I'm trying to find a single line of code recursively using powershell.
To look for the line "TODO" in a known file I can do:
get-content ActivityLibrary\Accept.cs | select-string TODO
But I don't want to explicitly type every directory\file. I would like to pipe a series of filenames from get-childitem like this:
gci -filter *.cs -name -recurse | gc | select-string TODO
But then I see this error:
Get-Content : The input object cannot be bound to any parameters for
the comman d either because the command does not take pipeline input
or the input and its properties do not match any of the parameters
that take pipeline input. At line:1 char:37
What am I doing wrong?
You need to remove the -Name switch. It outputs just file names, not file objects. And you can also pipe directly to Select-String and drop 'gc'.
Related
Trying to use select-string on a directory results in an error:
PS C:\> select-string pattern P:\ath\to\directory
select-string : The file P:\ath\to\directory cannot be read: Access to the path 'P:\ath\to\directory' is denied.
However, when I use get-childItem -recurse, the command finishes without problem:
PS C:\> get-childItem -recurse P:\ath\to | select-string pattern
This sursprises me because get-childItem recurse also passes directories down the pipeline. So, I'd have assumed that select-string raises the same error when it processes the first directory.
Since this is not the case, I wonder where or how the directory is filtered out in the pipeline.
TL;DR: object magic.
When Get-Childitem is run, it will return a collection of objects. This is passed to Select-String. The cmdlet's source is available on Github. There's a ProcessRecord() method that, well, processes input objects. It contains a few checks for object type and if those are directories. Like so,
if (_inputObject.BaseObject is FileInfo fileInfo)
...
if (expandedPathsMaybeDirectory && Directory.Exists(filename))
...
Thus, it doesn't matter that Get-Child's collection contains both DirectoryInfo and FileInfo objects; the cmdlet is smart enough to figure out what it's trying to consume.
The -Path parameter of Select-String expects paths to files:
-Path
Specifies the path to the files to search. Wildcards are permitted. The default location is the local directory.
Specify files in the directory, such as log1.txt, *.doc, or .. If you specify only a directory, the command fails.
That's why you get an error, when you pass a path to a directory to the -Path parameter.
When you pipe objects to Select-String, they do not necessarily need a Path attribute that can be mapped to -Path of Select-String:
You can pipe any object that has a ToString method to Select-String.
So you can also pipe raw strings to Select-String:
'test' | Select-String -Pattern 'test'
This line will return test.
Get-ChildItem returns objects of type System.IO.FileInfo or System.IO.DirectoryInfo. Select-String will use the Path attribute of System.IO.FileInfo to parse the content of the file and it will use the string representation of System.IO.DirectoryInfo to parse exactly this string representation (the path itsself). That's why you don't get errors in your pipeline.
New-Item -Name 'test' -Type Directory | Select-String -Pattern 'test'
This line will return the path to the just created directory test.
I have a list of EDI text files with specific text in them. Currently in order for our custom scripting to convert them into an SQL table, we need to be able to see the X12 file type in the filename. Because we are using SQL script to get the files into tables this solution needs to be a one line solution. We have a definition table of client files which specify which field terminator and file types to look for so we will be later substitute those values into the one line solution to be executed individually. I am currently looking at Powershell (v.3) to do this for maximum present and future compatibility. Also, I am totally new to Powershell, and have based my script generation on posts in this forum.
Files example
t.text.oxf.20170815123456.out
t.text.oxf.20170815234567.out
t.text.oxf.20170815345678.out
t.text.oxf.20170815456789.out
Search strings to find within files: (To find EDI X12 file type uniquely, which may be duplicated within the same file n times)
ST*867
ST*846
ST~867
ST~846
ST|867
ST|846
Here is what I have so far which does not show itself doing anything with the whatif parameter:
(Get-ChildItem .\ -recurse | Select-String -pattern 'ST~867' -SimpleMatch).Path | Foreach -Begin {$i=1} -Process {Rename-Item -LiteralPath $_ -NewName ($_ -replace 'out$','867.out' -f $i++) -whatif}
The fist part:
(Get-ChildItem .\ -recurse | Select-String -pattern 'ST~867' -SimpleMatch).Path
Simply gets a list of paths that we need to input to be renamed
The second part after the | pipe:
Foreach -Begin {$i=1} -Process {Rename-Item -LiteralPath $_ -NewName ($_ -replace '\.out','.867.out' -f $i++) -whatif}
will supposedly loop through that list and rename the files adding the EDI type to the end of the file. I have tried 'out$','867.out' with no change.
Current Errors:
The first part shows duplicated path elements probably because there are multiple Transaction Set Headers in the files, is there any way to force it to be unique?
The command does not show any Errors (red text) but with the whatif parameter shows that it does not rename any files (tried running it without as well).
1) remove duplicates using -List switch in Select-String
2) you need to really pipe the objects into the for loop
Try this?
Select-String -Path .\*.out -pattern 'ST~867' -SimpleMatch -List | Select-Object Path | ForEach-Object { Rename-Item $_.path ($_.path -replace 'out$','867.out') }
I have been reading through some of the previous posts about the concept, but all the solutions i find very different from eachother.
I have multiple csv's devided over seperate folders (all with kind of the same path but the subfolders are different).
I now need to import these csv's, change the headers and export it into one single csv.
I have been trying with this but im getting a very weird error: Import-Csv : Cannot open file "C:\Windows\system32\Book1.csv"
Although my path is refering to C:\csv ?
$CSVFolder = 'C:\csv\'; #'
$OutputFile = 'C:\export\test3.csv';
$CSV= #();
Get-ChildItem -Path $CSVFolder -Filter *.csv | ForEach-Object {
$CSV += #(Import-Csv -Path $_)
}
$CSV | Export-Csv -Path $OutputFile -NoTypeInformation -Force;
I was thinking of using a datatable for the headers, but its not so clear to me at this point. Also the issue with the error is not clear for me...
As has been noted:
Import-Csv -Path $_ should be Import-Csv -Path $_.FullName in your code,
(Strictly speaking, it should be Import-Csv -LiteralPath $_.FullName, because strings passed to the -Path parameter are subject to wildcard resolution).
because the [System.IO.FileInfo] instances representing files returned by Get-ChildItem are converted to strings when they are passed to the -Path (or -LiteralPath) parameter as a command-line argument (as opposed to via the pipeline), in which case the the mere file name rather than the full path is used, if your Get-ChildItem command targets a directory in Windows PowerShell (see background information below).
A mere filename such as Book1.csv is interpreted as relative to the current directory (which happened to be C:\Windows\system32 in your case), so Import-Csv looks for file C:\Windows\system32\Book1.csv rather than the intended C:\csv\Book1.csv.
Note that piping Get-ChildItem output to cmdlets is generally not affected by this, because the .PSPath property (which PowerShell adds behind the scenes) containing the full path (including PS provider prefix) binds to the -LiteralPath parameter.
Note that as of PSv5.1.14393.693, however, this mechanism is broken for Import-Csv, due to a bug.
This is a common pitfall that occurs whenever [System.IO.FileInfo] instances are passed to cmdlets that accept file paths via [string](-array)-typed parameters as arguments.
To be safe: Always use .FullName when you pass objects received from Get-ChildItem to another command as a parameter value (as opposed to via the pipeline) to ensure that the full path is passed.
Optional background information:
This behavior is a pitfall, because it is perfectly reasonable to assume that passing a [System.IO.FileInfo] instance as-is to a command that accepts file paths works, given the object-oriented nature of PowerShell - especially, since it does work reliably when using the pipeline rather than a parameter value.
Unfortunately, the built-in cmdlets that accept file paths (-Path, -LiteralPath parameters) do so as [string]s only (there is no parameter set that accepts [System.IO.FileInfo] instances directly), and it is in the course of [System.IO.FileInfo]-to-string conversion that the problem arises.
There also wouldn't be a problem if the [System.IO.FileInfo] instances consistently evaluated to the files' full paths, which is unfortunately not the case in Windows PowerShell (this has since been fixed in PowerShell Core):
Get-ChildItem <directory> outputs [System.IO.FileInfo] instances that evaluate to file names only in a string context.
Get-ChildItem <literalFilePathOrWildCardExpr> outputs [System.IO.FileInfo] instances that evaluate to full paths.
In other words: It is only if Get-ChildItem targets a directory (folder) that the objects it returns evaluate to their file names only in a string context.
Targeting a specific file or using a wildcard expression results in full paths, by contrast; with Get-Item, that's always the case.
You simply need to 'fullname' property, instead of 'name'.
Ex:
PS /Users/adil/Downloads> gi *csv |select name
Name
----
tradesdownload.csv
PS /Users/adil/Downloads> gi *csv |select name, fullname
Name FullName
---- --------
tradesdownload.csv /Users/adil/Downloads/tradesdownload.csv
try this code. This code take all csv file, import them and take only column 1, 2, 3 and change column name to header1, header2, header3, then export all into new csv file
Get-ChildItem "C:\temp2" -Filter "*.csv" |
%{Import-Csv $_.FullName -Header header1, header3,header4} |
Export-Csv "c:\temp\result.csv" -NoTypeInformation
#a short version (for no purist)
gci "C:\temp2" -Filter "*.csv" | %{ipcsv $_.FullName -Header header1, header3,header4} | epcsv "c:\temp\result.csv" -NoType
I have several directories, subdirectories, etc in a directory structure and some of them will have various matching files. e.g. if X.config.default exists in a directory it will also have a corresponding X.config.build
c:\Stuff\dir1
web.config.default
web.config.build
c:\Stuff\dir2
app.config.default
app.config.build
c:\Stuff\dir2\sub2
foo.config.default
foo.config.build
bar.config.default
bar.config.build
This will display all file names matching *.config.default and their corresponding directory
get-childitem -Recurse *.* -Filter *.config.default | Select Name, Directory
But instead of displaying the files and their path, I want to do something for each "match". In this case I want to call a program called ctt and send it three arguments. ctt is called as follows:
ctt s:<source file> t:<transform file> d:<destination file>
Assume the first match is called fubar in directory c:\Stuff\dir1, the ctt command executed should look like:
ctt s:c:\Stuff\dir1\fubar.config.default t:c:\Stuff\dir1\fubar.config.build d:c:\Stuff\dir1\fubar.config pw
I'm guessing there are a few ways of doing this. Piping get-childitem results into a command, or sending them to some sort of collection on which I can do a foreach loop.
Thanks
There are a few different ways of approaching this. If you're on an older version of PowerShell, you'd most likely just use the ForEach-Object cmdlet.
Get-ChildItem -Path c:\Stuff\* -Recurse -Filter *.config.default |
ForEach-Object -Process {
$BuildName = $PSItem.Name.Split('.')[0] ### Get just the "fubar" part.
ctt s:"$($PSItem.FullName)" t:"$($PSItem.Directory.FullName)\$BuildName.config.build" d:"$($PSItem.Directory.FullName).config" pw
}
On newer versions of PowerShell, starting with 4.0, you can use the ForEach() method syntax.
http://social.technet.microsoft.com/wiki/contents/articles/26489.powershell-4-0-where-and-foreach-method-syntax.aspx
(Get-ChildItem -Path c:\Stuff\* -Recurse -Filter *.config.default).ForEach({
$BuildName = $PSItem.Name.Split('.')[0] ### Get just the "fubar" part.
ctt s:"$($PSItem.FullName)" t:"$($PSItem.Directory.FullName)\$BuildName.config.build" d:"$($PSItem.Directory.FullName).config" pw
}
Searching the web for a findstr equivalent for Powershell I found this site, which suggests using the the Cmdlet gci (get child items) and select-string. However, gci doesn't print the content of a file, instead it prints the directory content. How the pipelining in this case works, how can gci and select-string filter the content of a file (without piplining it first to the get-content)?
Select-String accepts pipeline input. When you pipe FileInfo objects, they bind to the InputObject parameter. The following two commands are equivalent:
PS> Get-ChildItem C:\test.txt | Select-String -Pattern logfile
PS> Select-String -InputObject (Get-ChildItem C:\test.txt) -Pattern logfile
The select-string cmdlet recieves a System.IO.FileInfo object from the pipeline. Thus it is able to determine which part of its parameters are file names and which are the strings to look for. See Select-string at Technet.