piping get-childitem into select-string in powershell - powershell

I am sorting a large directory of files and I am trying to select individual lines from the output of an ls command and show those only, but I get weird results and I am not familiar enough with powershell to know what I'm doing wrong.
this approach works:
ls > data.txt
select-string 2012 data.txt
rm data.txt
but it seems wasteful to me to create a file just to read the data that I already have to fill into the file. I want to pipe the output directly to select-string.
I have tried this approach:
ls | select-string 2012
but that does not give me the appropriate output.
My guess is that I need to convert the output from ls into something select-string can work with, but I have no idea how to do that, or even whether that is actually the correct approach.

PowerShell is object-oriented, not pure text like cmd. If you want to get fileobjects(lines) that were modified in 2012, use:
Get-ChildItem | Where-Object { $_.LastWriteTime.Year -eq 2012 }
If you want to get fileobjects with "2012" in the filename, try:
Get-ChildItem *2012*
When you use
ls | select-string 2012
you're actually searching for lines with "2012" INSIDE every file that ls / get-childitem listed.
If you really need to use select-string on the output from get-childitem, try converting it to strings, then splitting up into lines and then search it. Like this:
(Get-ChildItem | Out-String) -split "`n" | Select-String 2012

I found another simple way to convert objects to strings:
Get-ChildItem | Out-String -stream | Select-String 2012
in this very interesting article:
http://blogs.msdn.com/b/powershell/archive/2006/04/25/how-does-select-string-work-with-pipelines-of-objects.aspx
If you wanted Select-String to work on the Monad formatted output, you'll need to get that as a string. Here is the thing to grok about
our outputing. When your command sequence emits a stream of strings,
we emit it without processing. If instead, your command sequence
emits a stream of objects, then we redirect those objects to the
command Out-Default. Out-Default looks at the type of the object and
the registered formating metadata to see if there is a default view
for that object type. A view defines a FORMATTER and the metadata for
that command. Most objects get vectored to either Format-Table or
Format-List (though they could go to Format-Wide or Format-Custom).
THESE FORMATTERS DO NOT EMIT STRINGS! You can see this for yourself
by the following: "These formating records are then vectored to an
OUT-xxx command to be rendered into the appropriate data for a
particular output device. By default, they go to Out-Host but you can
pipe this to Out-File, Out-Printer or Out-String. (NOTE: these
OUT-xxx commands are pretty clever, if you pipe formating objects to
them, they'll render them. If you pipe raw object to them, they'll
first call the appropriate formatter and then render them.)

Related

Powershell: how to retrieve powershell commands from a csv and execute one by one, then output the result to the new csv

I have a Commands.csv file like:
| Command |
| -----------------------------------------------|
|(Get-FileHash C:\Users\UserA\Desktop\File1).Hash|
|(Get-FileHash C:\Users\UserA\Desktop\File2).Hash|
|(Get-FileHash C:\Users\UserA\Desktop\File3).Hash|
Header name is "Command"
My idea is to:
Use ForEach ($line in Get-Content C:\Users\UserA\Desktop\Commands.csv ) {echo $line}
Execute $line one by one via powershell.exe, then output a result to a new .csv file - "result.csv"
Can you give me some directions and suggestions to implement this idea? Thanks!
Important:
Only use the technique below with input files you either fully control or implicitly trust to not contain malicious commands.
To execute arbitrary PowerShell statements stored in strings, you can use Invoke-Expression, but note that it should typically be avoided, as there are usually better alternatives - see this answer.
There are advanced techniques that let you analyze the statements before executing them and/or let you use a separate runspace with a restrictive language mode that limits what kinds of statements are allowed to execute, but that is beyond the scope of this answer.
Given that your input file is a .csv file with a Commands column, import it with Import-Csv and access the .Commands property on the resulting objects.
Use Get-Content only if your input file is a plain-text file without a header row, in which case the extension should really be .txt. (If it has a header row but there's only one column, you could get away with Get-Content Commands.csv | Select-Object -Skip 1 | ...). If that is the case, use $_ instead of $_.Commands below.
To also use the CSV format for the output file, all commands must produce objects of the same type or at least with the same set of properties. The sample commands in your question output strings (the value of the .Hash property), which cannot meaningfully be passed to Export-Csv directly, so a [pscustomobject] wrapper with a Result property is used, which will result in a CSV file with a single column named Result.
Import-Csv Commands.csv |
ForEach-Object {
[pscustomobject] #{
# !! SEE CAVEAT AT THE TOP.
Result = Invoke-Expression $_.Commands
}
} |
Export-Csv -NoTypeInformation Results.csv

How to extract a value out of a file, and save it in a new file, using Powershell

I recently started using Powershell and I'm trying out some code.
I have a .cfg file with several rules of code. The code is written like this:
ad.name=1
ad.virtual=active
ad.set=none
ad.partition=78
Now I want to export the value of ad.partition, which is 78, to a new file. I don't want to export ad.partition or = but only the number 78.
So far I got this:
Get-Content -Path C:\file.cfg | Where-Object {$_ -like 'ad.partition=78'}
But then I -obviously- just get the variable and the value. Not sure how to continue...
I hope someone has a way of achieving what I want.
After saving the value in a new file, would it be possible to add spaces? For example, the value consists out of 9 digits, e.g. 123456789. The desired output result would be 123 456 789.
Use ConvertFrom-StringData cmdlet to create a hash table from your file, then simply index the key you are after:
$h=(Get-Content -Path C:\file.cfg | ConvertFrom-StringData)
$h.("ad.partition") -replace ('^(\d{1,3})(\d{1,3})?(\d{1,3})?','$1 $2 $3') > C:\out.cfg
You can use the Select-String cmdlet to capture your desired value using a regex. Then just pipe the result to the Out-File cmdlet. To get your desired output with spaces, you can use a simple format string:
"{0:### ### ###}" -f [int](Select-string 'ad\.partition=(.*)' -Path C:\file.cfg).Matches.Groups[1].Value |
Out-File C:\result.cfg

"$xyz" and "Write-Host "$xyz"" giving different output

I am hashing all the files in one location, an origin folder, and writing the hashes to a variable and then doing the same to all the files in another location, a destination folder:
$origin = Get-ChildItem .\Test1 | Get-FileHash | Format-Table -Property Hash -HideTableHeaders
$destination = Get-ChildItem .\Test2 | Get-FileHash | Format-Table -Property Hash -HideTableHeaders
Then I am comparing them with Compare-Object like so:
Compare-Object $origin $destination
Now in my test I purposefully have deviations, so when the above code returned no differences I knew I had a problem.
Then I found out that if I do the following, that the hash values arn't there:
PS> Write-Host "$origin"
Microsoft.PowerShell.Commands.Internal.Format.FormatStartData Microsoft.PowerShell.Commands.Internal.Format.GroupStartData Microsoft.PowerShell.Commands.Internal.Format.FormatEntryData Microsoft.PowerShell.Commands.Internal.Format.FormatEntryData Microsoft.PowerShell.Commands.Internal.Format.FormatEntryData Microsoft.PowerShell.Commands.Internal.Format.GroupEndData Microsoft.PowerShell.Commands.Internal.Format.FormatEndData
However, if I just type the following and press enter, then the hash values are present (like I want):
PS> $origin
6B86B273FF34FCE19D6B804EFF5A3F5747ADA4EAA22F1D49C01E52DDB7875B4B
D4735E3A265E16EEE03F59718B9B5D03019C07D8B6C51F90DA3A666EEC13AB35
4E07408562BEDB8B60CE05C1DECFE3AD16B72230967DE01F640B7E4729B49FCE
I am assuming when I use Compare-Object, that my variables are not presenting the hash values like I expected.
Does anyone know what is going on or have any recommendations? This is being used to ensure files are moved from an origin location to a destination location (this is one check in a script I'm working on). I am keeping this purely PowerShell, which means no xcopy or robocopy.
Re use of Format-Table to create the input collections for Compare-Object:
Only ever use Format-* cmdlets for display formatting; never use them if data must be programmatically processed.
Format-* cmdlets output formatting instructions, not data - see this answer.
Therefore:
Omit the Format-Table calls from your input-collection definition commands:
$origin=Get-ChildItem .\Test1 | Get-FileHash
$destination=Get-ChildItem .\Test2 | Get-FileHash
Then pass the names of the properties to compare the objects by to Compare-Object:
Compare-Object $origin $destination -Property Path, Hash
Note the need to compare by both path and hash, to make sure that only files of the same name are compared.
As an aside: If you didn't specify -Property, the objects would by default be compared by their .ToString() value - and since the Microsoft.PowerShell.Commands.FileHashInfo instances output by Get-FileHash only ever stringify to that very type name (irrespective of their specific property values), no differences would be found.
As for $origin vs. Write-Host $orgin:
Just executing $origin is implicitly like executing Write-Output $origin - it writes to the success output stream (see about_Redirection), which by default goes to the console.
Success output that goes to the console is automatically formatted by PowerShell's rich output-formatting system.
Write-Host, by contrast, serves a different purpose than Write-Output:
It writes directly to the console[1], bypassing PowerShell's success output stream and thereby also its usual formatting. Its primary purpose is to write status messages, interactive prompt messages, ... to the display - as opposed to outputting data.
Write-Host itself applies output formatting, but only by simple .ToString() stringification, which often yields unhelpful (type name-only) representations, as in your case.
See this answer for more information about the differences between Write-Output and Write-Host.
[1] Technically, since PowerShell version 5, Write-Host output reaches the console via the information output stream (number 6), but its primary purpose is still to write to the display as opposed to outputting data.

Select-String in Powershell only displaying part of the line from a text file, need it to display whole thing

I am trying to write a simple PS script to check large .txt log files for a short string: "SRVE0242I:"
$lines = Select-String -Path $logDir -Pattern "SRVE0242I:" | Select-Object line | Out-String
On output though, it only displays the following:
Line
[28/06/17 13:48:27:839] 00000020 ServletWrappe I SRVE0242I: [User] [User] [com_xxxxxxx_...
And not the full line. Is there a limit to how many characters this pulls? I can't find any info on any restrictions for the Select-String cmdlet. Is there a better way to do this so that I don't a) pull the heading "Line" in my list of lines (Don't really want to create table formatting for such a simple output) and b) get the whole line when I pull the info?
You are seeing it like this because it's displaying the Line property using the default Format-Table view and shortening it to the width of the console.
Do this instead:
$lines = Select-String -Path $logDir -Pattern "SRVE0242I:" | Select-Object -ExpandProperty line
This returns the value of the Line property as a string to the $lines variable. You don't need to use Out-String.
There is! Long story short, Select-Object is doing the truncating here. Here's one way to get the first untruncated line in a Select-String output
$(Select-String -Path $logDir -Pattern "SRVE0242I:")[0].Line
When you run into something like this, you can break down the individual steps to determine what's happening by piping things to Get-Member. Here's what's happening in the code above:
Select-String <# args #> | Get-Member
Select-String gives us a MatchInfo object, which (as you've correctly determined) has a 'Line' property. When run on it's own, Select-String will actually spit out all the information you're looking for, and will not truncate it by default (at least, on v6.0.0-beta). It does give you an array of MatchInfo objects if it finds multiple matches, so you have to index into that array if you just want the first one (like I did above).
Select-String <# args #> | Select-Object Line | Get-Member
Select-Object applies PowerShell's default formatting for objects which, in most cases, will truncate your output for easier viewing. For objects with a bunch of members (like a MatchInfo object), it will try to do one per line by default.
Select-String <# args #> | Select-Object Line | Out-String | Get-Member
Out-String directly translates it's input to a string. That is, rather than trying to cast something to a string or pull a string Property out of an object that's passed to it, it just changes whatever it receives into an object. In this case, it turns the already-formatted MatchInfo output into a string. Nothing happens to the output on the terminal, but Get-Member will reveal a String rather than a MatchInfo object.
It's not directly relevant here, but if you're interested in modifying the default formatting, it's governed by the types.ps1xml file.

How can I filter filename patterns in PowerShell?

I need to do something similar to Unix's ls | grep 'my[rR]egexp?' in Powershell. The similar expression ls | Select-String -Pattern 'my[rR]egexp?' seems to go through contents of the listed files, rather than simply filtering the filenames themselves.
The Select-String documentation hasn't been of much help either.
Very simple:
ls | where Name -match 'myregex'
There are other options, though:
(ls) -match 'myregex'
Or, depending on how complex your regex is, you could maybe also solve it with a simple wildcard match:
ls wild[ck]ard*.txt
which is faster than above options. And if you can get it into a wildcard match without character classes you can also just use the -Filter parameter to Get-ChildItem (ls), which performs filtering on the file system level and thus is even faster. Note also that PowerShell is case-insensitive by default, so a character class like [rR] is unnecessary.
While researching based on #Joey's answer, I stumbled upon another way to achieve the same (based on Select-String itself):
ls -Name | Select-String -Pattern 'my[Rr]egexp?'
The -Name argument seems to make ls return the result as a plain string rather than FileInfo object, so Select-String treats it as the string to be searched in rather than a list of files to be searched.