The original code was this:
curl -sL https://ftp.apnic.net/stats/apnic/delegated-apnic-latest | \
grep "apnic|JP|ipv4" | \
awk -F '|' '{ printf("%s/%d\n", $4, 32-log($5)/log(2)) }' | \
tee JP_IPv4.txt
I wanted to convert it to PowerShell, this is how far I got:
Invoke-WebRequest -Uri "https://ftp.apnic.net/stats/apnic/delegated-apnic-latest" |
Select-String -Pattern "afrinic\|ZA\|ipv6" -ca | Select-Object -exp line |
I need to figure out how to convert the last part that uses awk command to PowerShell.
I use PowerShell 7.3
The awk script appears to calculate the subnet prefix length from the network size.
You can replace the awk log(...) function call with [Math]::Log(...) in PowerShell to perform the exact same calculation:
# this takes care of the `curl` part
Invoke-WebRequest -Uri "https://ftp.apnic.net/stats/apnic/delegated-apnic-latest" -OutFile delegated_networks.txt
# this takes care of the `grep` part
Get-Content delegated_networks.txt |Where-Object {$_ -like 'apnic|JP|ipv4|*'}
# this emulates the `awk | tee` part
$data|ForEach-Object {
# -F '|'
$cells = $_.Split('|')
# '{ printf("%s/%d\n", $4, 32-log($5)/log(2)) }'
'{0}/{1}' -f $cells[3],(32 - [Math]::Log($cells[4])/[Math]::Log(2))
} |Tee-Object -FilePath JP_ipv4.txt
As #mklement0 kindly points out, you can also combine the log-conversion operation into a single call in PowerShell:
PS ~>[Math]::Log(4096)/[Math]::Log(2)
12
PS ~>[Math]::Log(4096, 2)
12
Related
I need to search for a term in a file and display an index in the search results so that referencing is easy.
The command in bash would be:
cat <file> | grep 'Table: ' | cat -n
What I have so far in Powershell:
Get-Content <file> | Select-String 'Table: ' | Select-Object -Property LineNumber, Line
Unfortunately, I didn't realize that LineNumber gives the actual line in the file and not the index in the results list.
How can I translate the bash command into its Powershell equivalent?
Indeed, the .Line property of the objects output by Select-Object indicates the line number of each match in a given input file.
PowerShell has no direct equivalent of cat -n (prepending a 1-based index to all input lines on output), but it's not hard to roll your own using the ForEach-Object cmdlet:
$i = 0
Get-Content file.txt | Select-String 'Table: ' | ForEach-Object {
"{0,6}`t{1}" -f ++$i, $_.Line
}
The above uses -f, the format operator, to left-space-pad to 6 characters (,6) the first RHS operand ({0}), which is the (incremented) index, ++$i, followed by a tab character (`t) and the second RHS operand ({1}), which is the input line at hand ($_.Line).
Just use WSL:
bash -c "cat <file> | grep 'Table: ' | cat -n"
This will run the bash code in powershell. For a true powershell option you could do this:
foreach ($line in $(Get-Content -Path <filepath> | Select-String 'Table: ')){
$count++
echo "$count $line"
}
I have a json config that I want to sed in Powershell. Here is the entry in config.json:
"gateway.public.host": "http://placeholder.ourdomain.com",
I want to find the line matching the string gateway.public.host but I want to replace the word placeholder on that line.
Most examples I find on Powershell is find a match and replace that match.
A regex approach would be obvious here. Match everything from the beginning of a line up until and including "placeholder" with a capture group replacing the text after the capture group leaving the remaining line intact.
(Get-Content $file) -replace "(.*gateway.public.host.*)placeholder", '$1ReplacementText'
A host of other regexes will work here as well.
However we can also leverage the power in PowerShell with the cmdlets ConvertFrom-Json and ConvertTo-Json
$json = '{
"gateway.public.host": "http://placeholder.ourdomain.com"
}'
$jsonObject = $json | ConvertFrom-Json
$jsonObject."gateway.public.host" = $jsonObject."gateway.public.host" -replace "placeholder", "holdplacer"
$jsonObject | ConvertTo-Json -Depth 5
Which will get you
{
"gateway.public.host": "http://holdplacer.ourdomain.com"
}
Yes, I will concede that there is more code there. Depending on what you are starting with and where you want to end up your code will be cleaner using this method.
Replace is kind of the SED equivalent. Seeing as you're new to Powershell I'd go with the first method.
If your scenario really is that specific:
$Json = Get-Content filename.json
$Json = $Json -replace ("`"gateway.public.host`": `"http://placeholder.ourdomain.com`"","`"gateway.public.host`": `"http://newvalue.ourdomain.com`"")
$Json | Set-Content filename.json -Force
Or you can do it in one-line
(Get-Content filename.json).replace("`"gateway.public.host`": `"http://placeholder.ourdomain.com`"","`"gateway.public.host`": `"http://newvalue.ourdomain.com`"") | Set-Content filename.json
In Powershell cat is an alias for get-content so you could even do this if you want to 'feel' more linux:
cat filename.json | %{$_ -replace "`"gateway.public.host`": `"http://placeholder.ourdomain.com`"","`"gateway.public.host`": `"http://newvalue.ourdomain.com`""} | Set-Content filename.json
We have the following unix command:
/usr/bin/tail -n 1 %{path} | grep --silent -F "%{message}" && rm -f %{path}%
This:
/usr/bin/tail -n 1 %{path} gets the last line in the file that the path variable refers to
| grep --silent -F "%{message}" pipes the output to another command, grep, which checks if the output of the previous command is equal to the value of message
&& rm -f %{path}% if the values are equal, then delete the file refered to by path
The above line is in a configuration file which is allows for calls to be made to the underlying operating system.
I want to replicate the functionalirty on windows.
I tried this:
command => 'powershell -Command "& {Get-Item $args[0] | ? { (Get-Content $_ -Tail 1).Contains($args[1]) }| Remove-Item -Force}" "'%path%'" "'%message%'"'
This error is thrown:
Error: Expected one of #, {, } at line 15, column 131 (byte 498)
Line 15 is the line in the configuration file which contains the above.
Thanks
PowerShell solution:
$path = 'C:\path\to\your.txt'
$message = 'message'
Get-Item $path | ? { (Get-Content $_ -Tail 1).Contains($message) } | Remove-Item -Force
If you want to run it from a command line, call it like this:
powershell -Command "& {Get-Item $args[0] | ? { (Get-Content $_ -Tail 1).Contains($args[1]) } | Remove-Item -Force}" "'C:\path\to\your.txt'" "'message'"
You can use tailhead.bat (pure batch script utility) that can be used to show lasts/fists lines of a file.Instead of Grep you can use findstr or find :
tailhead.bat tailhead -file=%pathToFile% -begin=-3|find "%message%"
I'm looking for the PowerShell equivalent to grep --file=filename. If you don't know grep, filename is a text file where each line has a regular expression pattern you want to match.
Maybe I'm missing something obvious, but Select-String doesn't seem to have this option.
The -Pattern parameter in Select-String supports an array of patterns. So the one you're looking for is:
Get-Content .\doc.txt | Select-String -Pattern (Get-Content .\regex.txt)
This searches through the textfile doc.txt by using every regex(one per line) in regex.txt
PS) new-alias grep findstr
PS) C:\WINDOWS> ls | grep -I -N exe
105:-a--- 2006-11-02 13:34 49680 twunk_16.exe
106:-a--- 2006-11-02 13:34 31232 twunk_32.exe
109:-a--- 2006-09-18 23:43 256192 winhelp.exe
110:-a--- 2006-11-02 10:45 9216 winhlp32.exe
PS) grep /?
I'm not familiar with grep but with Select-String you can do:
Get-ChildItem filename.txt | Select-String -Pattern <regexPattern>
You can also do that with Get-Content:
(Get-Content filename.txt) -match 'pattern'
I had the same issue trying to find text in files with powershell. I used the following - to stay as close to the Linux environment as possible.
Hopefully this helps somebody:
PowerShell:
PS) new-alias grep findstr
PS) ls -r *.txt | cat | grep "some random string"
Explanation:
ls - lists all files
-r - recursively (in all files and folders and subfolders)
*.txt - only .txt files
| - pipe the (ls) results to next command (cat)
cat - show contents of files comming from (ls)
| - pipe the (cat) results to next command (grep)
grep - search contents from (cat) for "some random string" (alias to findstr)
Yes, this works as well:
PS) ls -r *.txt | cat | findstr "some random string"
So I found a pretty good answer at this link:
https://www.thomasmaurer.ch/2011/03/powershell-search-for-string-or-grep-for-powershell/
But essentially it is:
Select-String -Path "C:\file\Path\*.txt" -Pattern "^Enter REGEX Here$"
This gives a directory file search (*or you can just specify a file) and a file-content search all in one line of PowerShell, very similar to grep. The output will be similar to:
doc.txt:31: Enter REGEX Here
HelloWorld.txt:13: Enter REGEX Here
I find out a possible method by "filter" and "alias" of PowerShell, when you want use grep in pipeline output(grep file should be similar):
first define a filter:
filter Filter-Object ([string]$pattern) {
Out-String -InputObject $_ -Stream | Select-String -Pattern "$pattern"
}
then define the alias:
New-Alias -Name grep -Value Filter-Object
final, put the former filter and alias in your profile:
$Home[My ]Documents\PowerShell\Microsoft.PowerShell_profile.ps1
Restart your PS, so you can use it:
alias | grep 'grep'
References
alias:
Set-Alias here
New-Alias here
Filter (Special function) here
Profiles (just like .bashrc for bash): here
out-string (this is the key) here:
in PowerShell Output is object-based hereļ¼so the key
is to convert object to string and grep the string.
Select-String here:
Finds text in strings and files
This question already has an answer, but I just want to add that in Windows there is Windows Subsystem for Linux WSL.
So for example if you want to check if you have service named Elasicsearch that is in status running you can do something like the snippet below in powershell
net start | grep Elasticsearch
but select-String doesn't seem to have this option.
Correct. PowerShell is not a clone of *nix shells' toolset.
However it is not hard to build something like it yourself:
$regexes = Get-Content RegexFile.txt |
Foreach-Object { new-object System.Text.RegularExpressions.Regex $_ }
$fileList | Get-Content | Where-Object {
foreach ($r in $regexes) {
if ($r.IsMatch($_)) {
$true
break
}
}
$false
}
Maybe?
[regex]$regex = (get-content <regex file> |
foreach {
'(?:{0})' -f $_
}) -join '|'
Get-Content <filespec> -ReadCount 10000 |
foreach {
if ($_ -match $regex)
{
$true
break
}
}
I'm trying to do some processing on the output from a psftp "ls" command. Basically, I want to put all of the file names that match certain conditions into an array. I have this:
PS C:\path\to\pwd> $a = & "C:\Program Files (x86)\PuTTY\psftp.exe" -l myusername -batch -pw mypassword -b sftpbatch.txt myserver | where {$_.split(" ", [StringSplitOptions]'RemoveEmptyEntries')[0] -eq "-rw-r--r--"} | select-object {$_.split(" ", [StringSplitOptions]'RemoveEmptyEntries')[8]}
(If you want more details about that command, I can provide them. The output is very similar to the output of the "ls" command in PowerShell.)
It seems to me that I can do better by selecting the split first, then filtering it with where. When I try this:
$a = & <# ... #> | select-object {$_.split(" ", [StringSplitOptions]'RemoveEmptyEntries')} | where { $_[0] -eq "-rw-r--r--" }
I get
Unable to index into an object of type System.Management.Automation.PSObject.
How can I simplify this?
Something like this should work.
$a = & <#...#>| % {$b = $_ -split ' '|?{$_}; if($b[0] -eq '-rw-r--r--'){$b[8]}}
If you are placing this in a script, I would replace the alias % with Foreach-Object, and the alias ? with Where-Object
Edit:
Here is a more pipeline-oriented approach:
'-rw-r--r-- 1 2 3 4 5 6 7 test.txt'|
select #{Name='Words'; Expression={$_ -split ' '|where {$_}}}|
where {$_.Words[0] -eq "-rw-r--r--"}|
foreach {$_.Words[8]}