How can I pass the results of get-childitem into a command? - powershell

I have several directories, subdirectories, etc in a directory structure and some of them will have various matching files. e.g. if X.config.default exists in a directory it will also have a corresponding X.config.build
c:\Stuff\dir1
web.config.default
web.config.build
c:\Stuff\dir2
app.config.default
app.config.build
c:\Stuff\dir2\sub2
foo.config.default
foo.config.build
bar.config.default
bar.config.build
This will display all file names matching *.config.default and their corresponding directory
get-childitem -Recurse *.* -Filter *.config.default | Select Name, Directory
But instead of displaying the files and their path, I want to do something for each "match". In this case I want to call a program called ctt and send it three arguments. ctt is called as follows:
ctt s:<source file> t:<transform file> d:<destination file>
Assume the first match is called fubar in directory c:\Stuff\dir1, the ctt command executed should look like:
ctt s:c:\Stuff\dir1\fubar.config.default t:c:\Stuff\dir1\fubar.config.build d:c:\Stuff\dir1\fubar.config pw
I'm guessing there are a few ways of doing this. Piping get-childitem results into a command, or sending them to some sort of collection on which I can do a foreach loop.
Thanks

There are a few different ways of approaching this. If you're on an older version of PowerShell, you'd most likely just use the ForEach-Object cmdlet.
Get-ChildItem -Path c:\Stuff\* -Recurse -Filter *.config.default |
ForEach-Object -Process {
$BuildName = $PSItem.Name.Split('.')[0] ### Get just the "fubar" part.
ctt s:"$($PSItem.FullName)" t:"$($PSItem.Directory.FullName)\$BuildName.config.build" d:"$($PSItem.Directory.FullName).config" pw
}
On newer versions of PowerShell, starting with 4.0, you can use the ForEach() method syntax.
http://social.technet.microsoft.com/wiki/contents/articles/26489.powershell-4-0-where-and-foreach-method-syntax.aspx
(Get-ChildItem -Path c:\Stuff\* -Recurse -Filter *.config.default).ForEach({
$BuildName = $PSItem.Name.Split('.')[0] ### Get just the "fubar" part.
ctt s:"$($PSItem.FullName)" t:"$($PSItem.Directory.FullName)\$BuildName.config.build" d:"$($PSItem.Directory.FullName).config" pw
}

Related

Powershell. Combine text files in folders across multiple directories

I have tried to do my research, but I can't fathom this one out.
I can combine multiple .txt files in a folder. no problem:
dir C:\Users\XXXX\AC1\22JUN *.txt | get-content | out-file C:\Users\XXXX\22JUN\AC1_22JUN.txt
however, I have 14 Directories each with subdirectories. (months of the year), and this will only ever grow. How can I write it so that it will go into each directory AC1 - AC14 and then look into each folder JAN-DEC and in each subdirectory create a combined file for AC1_22JUN, AC2_22JUN AC1_22JUL, AC2_22JUL and so on and so on?
is there also a way to rename the output file with data, such as the number of .txt files that have been combined. i.e. AC1_22JUN_314.txt
many thanks in advance
What you need to do is iterate over all your directories and their subdirectories and run a particular command in each of them. This is easy enough to achieve using the cmdlet Get-ChildItem and a pair of nested foreach loops.
In addition, you need to count the number of text files you've processed so that you can name your aggregate file appropriately. To do this you can break your pipeline using the temporary variable $files. You can later begin a new pipeline with this variable and use its count property to name the aggregate file.
The code for this is as follows:
$dirs = Get-ChildItem -Directory
foreach ($dir in $dirs)
{
$subdirs = Get-ChildItem $dir -Directory
foreach ($subdir in $subdirs)
{
$files = Get-ChildItem *.txt -Path $subdir.fullname
$name = "$($dir.name)_$($subdir.name)_$($files.count).txt"
$files | Get-Content | Out-File "$($subdir.fullname)/$name"
}
}
A few things to note:
The script needs to be run from the containing folder - in your case the parent folder for AC1-AC14. To run it from elsewhere you will have to change the first statement into something like $dirs = Get-ChildItem C:\path\to\container -Directory
Get-ChildItem is the same as the command dir. dir is an alias for Get-ChildItem. This is NOT the same as the variable $dir which I've used in the script.
Running the script multiple times will include any and all of your old aggregate files! This is because the output file is also a .txt file which is caught in your wildcard search. Consider refining the search criteria for Get-ChildItem, or save the output elsewhere.

Powershell how to get-content across several subfolders

I'm working on a script to output some data from multiple files based on a string search. It outputs the string found, followed by the following six characters. I can get this to work for an exact location. However, I want to search across files inside multiple subfolders in the path. Using the below script, I get PermissionDenied errors...
[regex] $pattern = '(?<=(a piece of text))(?<chunk>.*)'
Get-Content -Path 'C:\Temp\*' |
ForEach-Object {
if ($_ -match $pattern) {
$smallchunk = $matches.chunk.substring(0, 6)
}
}
"$smallchunk" | Out-File 'C:\Temp\results.txt'
If I change -Path to one of the subfolders, it works fine, but I need it to go inside each subfolder and execute the get-content.
e.g., look inside...
C:\Temp\folder1\*
C:\Temp\folder2\*
C:\Temp\folder3\*
And so on...
Following up on boxdog's suggestion of Select-String, the only limitation would be folder recursion. Unfortunately, Select-String only allows the searching of multiple files in one directory.
So, the way around this is piping the output of Get-ChildItem with a -Recurse switch into Select-String:
$pattern = "(?<=(a piece of text))(?<chunk>.*)"
Get-ChildItem -Path "C:\Temp\" -Exclude "results.txt" -File -Recurse |
Select-String -Pattern $pattern |
ForEach-Object -Process {
$_.Matches[0].Groups['chunk'].Value.Substring(0,6)
} | Out-File -FilePath "C:\Temp\results.txt"
If there's a need for the result to be saved to $smallchunk you can still do so inside the loop if need be.
Abraham Zinala's helpful answer is the best solution to your problem, because letting Select-String search your files' content is faster and more memory-efficient than reading and processing each line with Get-Content.
As for what you tried:
Using the below script I get PermissionDenied errors...
These stem from directories being among the file-system items output by Get-ChildItem, which Get-Content cannot read.
If your files have distinct filename extensions that your directories don't, one option is to pass them to the (rarely used with Get-Content) -Include parameter; e.g.:
Get-Content -Path C:\Temp\* -Include *.txt, *.c
However, as with Select-String, this limits you to a single directory's content, and it doesn't allow you to limit processing to files fundamentally, if extension-based filtering isn't possible.
For recursive listing, you can use Get-ChildItem with -Recurse, as in Abraham's answer, and pipe the file-info objects to Get-Content:
Get-ChildItem -Recurse C:\Temp -Include *.txt, *.c | Get-Content
If you want to simply limit output to files, whatever their name is, use the -File switch (similarly, -Directory limits output to directories):
Get-ChildItem -File -Recurse C:\Temp | Get-Content

Copying files to specific folder declared in a CSV file using Powershell Script

i am quite new to powershell and i am trying to make a script that copy files to certain folders that are declared in a CSV file. But till now i am getting errors from everywhere and can't find nothing to resolve this issue.
I have this folders and .txt files created in the same folder as the script.
Till now i could only do this:
$files = Import-Csv .\files.csv
$files
foreach ($file in $files) {
$name = $file.name
$final = $file.destination
Copy-Item $name -Destination $final
}
This is my CSV
name;destination
file1.txt;folderX
file2.txt;folderY
file3.txt;folderZ
As the comments indicate, if you are not using default system delimiters, you should make sure to specify them.
I also recommend typically to use quotes for your csv to ensure no problems with accidentally including an entry that includes the delimiter in the name.
#"
"taco1.txt";"C:\temp\taco2;.txt"
"# | ConvertFrom-CSV -Delimiter ';' -Header #('file','destination')
will output
file destination
---- -----------
taco1.txt C:\temp\taco2;.txt
The quotes make sure the values are correctly interpreted. And yes... you can name a file foobar;test..txt. Never underestimate what users might do. 😁
If you take the command Get-ChildItem | Select-Object BaseName,Directory | ConvertTo-CSV -NoTypeInformation and review the output, you should see it quoted like this.
Sourcing Your File List
One last tip. Most of the time I've come across a CSV for file input lists a CSV hasn't been needed. Consider looking at grabbing the files you in your script itself.
For example, if you have a folder and need to filter the list down, you can do this on the fly very easily in PowerShell by using Get-ChildItem.
For example:
$Directory = 'C:\temp'
$Destination = $ENV:TEMP
Get-ChildItem -Path $Directory -Filter *.txt -Recurse | Copy-Item -Destination $Destination
If you need to have more granular matching control, consider using the Where-Object cmdlet and doing something like this:
Get-ChildItem -Path $Directory -Filter *.txt -Recurse | Where-Object Name -match '(taco)|(burrito)' | Copy-Item -Destination $Destination
Often you'll find that you can easily use this type of filtering to keep CSV and input files out of the solution.
example
Using techniques like this, you might be able to get files from 2 directories, filter the match, and copy all in a short statement like this:
Get-ChildItem -Path 'C:\temp' -Filter '*.xlsx' -Recurse | Where-Object Name -match 'taco' | Copy-Item -Destination $ENV:TEMP -Verbose
Hope that gives you some other ideas! Welcome to Stack Overflow. 👋

PowerShell Script finding File using extension and excluding a folder

I am using the below code:
Get-ChildItem -Path N:\USERS -Filter DANTOM.DTM -Recurse -ErrorAction SilentlyContinue -Force
I need it to either find the file "DANTOM.DTM" or the extension ".DTM". I need to exclude the folder N:\USERS\EDI because it is a 1.7TB folder that would never have this file in it. So in doing so would really speed up the process.
I would like the end result to either spit into a .txt file saying which folders inside of N:\USERS has the file or just have it display as a list in powershell.
Thank you,
Assuming that the files of interest do not reside directly in N:\USERS (only in subdirs.), try the following (PSv3+ syntax); send to a file by appending > dirs.txt, for instance.
Get-ChildItem N:\USERS -Directory | ? Name -ne 'EDI' |
Get-ChildItem -Recurse -Filter *.DTM |
ForEach-Object { $_.DirectoryName }
Note: While it is tempting to try a simpler approach with -Exclude EDI, it unfortunately doesn't seem to be effective in excluding the entire subtree of the EDI subfolder.

Get-ChildItem -Exclude returning Fullname instead of short name

I'm having a strange issue when I run a Get-ChildItem -Exclude.
I've been using the following command at both source and destination locations to create an array of files after I've run a copy and then comparing the 2 directories. It has been working just fine. (I was going to post images, but don't have a high enough reputation)
$sourcefiles = #(Get-ChildItem -Recurse -path e:\Home\o365.test)
When I run this command, the array populates with the short name of the files.
I excluded PST files from the file copy and I needed to also exclude them in the compare, so I added an exclude to the GCI command.
$sourcefiles = #(Get-ChildItem -Recurse -path e:\Home\o365.test -Exclude *.pst)
This returns all of the correct files (*.pst is in fact excluded), however it is returning the fullname of the files, rather than the shortname that I always got returned before.
This is causing the compare-object to fail, since the destination GCI doesn't have the exclude (Not needed since *.pst was excluded from the copy). Besides, it is a UNC path and the fullname wouldn't match anyway.
I realize I could use split-path -leaf (which I've tried - and seems to work)
$sourcefiles = #(Get-ChildItem -Recurse -path e:\Home\o365.test -Exclude *.pst | Split-Path -Leaf)
But I still don't understand why adding the -exclude param to GCI causes it to change the format of what it returns.
Does anyone have any insight into why this would be happening?
Thanks in advance for any help!
Well, I can't replicate what you're seeing here. Placing an at sign before an operation in PowerShell is in fact the array operator, put simply putting it before a parenthesis with an operation doesn't return single properties like you're describing.
Now, you can make this happen if you place a property name on the back of the parenthesis to select just one property. For example:
$files = #(Get-ChildItem -path c:\temp).BaseName
>LZelda.csv
>msvcr100.dll
>names-0.txt
>names-1.txt
>NOBGW00017.log
>nslookup.exe
>PolicySpy.exe
Adding the -Exclude parenthesis doesn't change the output at all.
$files = #(Get-ChildItem -path c:\temp -exclude *.png)
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 3/18/2015 2:34 PM Android
da---- 9/10/2013 7:17 PM Bootfiles
d----- 12/13/2014 4:29 PM camera
d----- 1/6/2015 11:49 AM CFS
d----- 12/11/2014 1:08 PM ebay
d----- 2/2/2014 1:55 PM Finance
My reason for showing you this is that the things we get back are still FileSystem objects. The only way to reduce $files down to containing just one property, like the FullName, BaseName or whatever is in the following two methods:
$files = #(Get-ChildItem -path c:\temp -exclude *.png).BaseName
$files = #(Get-ChildItem -path c:\temp -exclude *.png | Select -Expand BaseName)
Both will give the same output.
The problem here is a deliberate "bug" (as I call it) introduced right from the start of Powershell (or at least from PS2.0). In order to ease the transition from cmd to Powershell, Get-ChildItem was given the alias dir and was then made to work like the cmd version of dir (as far as necessary to give a similar looking output). As with cmd,
PS > dir <directory path>
produces a listing of <directory path> (when simply displayed). In both cases (cmd and Powershell), an absent directory path uses . (the current directory). However, if the cmd version is invoked using a file name or wildcard pattern after the directory path (again, implicit . if no directory path) then the output is the matching file(s) (and directories for the wildcard case) in that directory (if any). Get-ChildItem does the same so that the alias, dir, gives similar output. It might have been expected that Get-ChildItem, when provided a wildcard path, would return a list of the children of any directories matching that wildcard path and ignore any matching files. If the argument were a plain file name, Get-ChildItem would complain that the argument provided cannot have children (as opposed to an empty directory which can have children but just doesn't). After all, if you want to match files or directories themselves (rather than their contents), you'd use Get-Item. Indeed, the results returned by Get-ChildItem in the case of file arguments and wildcards are identical to the equivalent Get-Item. However, the results returned by Get-ChildItem for actual children of a specified directory path differ slightly from those of Get-Item. Specifically, they have different ToString() methods (even though they both return FileInfo or DirectoryInfo objects). This means that converting the object to a string either by explicitly invoking ToString() or by using the returned object in an expression that requires a string, e.g.
PS > dir | foreach { "$_" }
gives different results.
Demonstration,
PS > gci C:\Windows\Web\Screen | foreach { "$_" }
img100.jpg
img101.png
img102.jpg
img103.png
img104.jpg
img105.jpg
PS > gci C:\Windows\web\screen\* | foreach { "$_" }
C:\Windows\web\screen\img100.jpg # note: uses argument path case.
C:\Windows\web\screen\img101.png # actual directory path case is
C:\Windows\web\screen\img102.jpg # C:\WINDOWS\Web\Screen
C:\Windows\web\screen\img103.png
C:\Windows\web\screen\img104.jpg
C:\Windows\web\screen\img105.jpg
PS > gi C:\Windows\Web\screen\* | foreach { "$_" }
C:\Windows\Web\screen\img100.jpg
C:\Windows\Web\screen\img101.png
C:\Windows\Web\screen\img102.jpg
C:\Windows\Web\screen\img103.png
C:\Windows\Web\screen\img104.jpg
C:\Windows\Web\screen\img105.jpg
PS > gci C:\Windows\Web\screen\img100.jpg | foreach { "$_" }
C:\Windows\Web\screen\img100.jpg
PS > gi C:\Windows\Web\screen\img100.jpg | foreach { "$_" }
C:\Windows\Web\screen\img100.jpg
Thus, I suspect that the OP is using implicit string conversion to get the "name" of the file and running foul of this "bug" which apparently also manifests when -Exclude is used (independent of -Recurse which has no effect). The solution is to not rely on string conversion and use the actual string properties, either name or fullname (whichever is needed).
PS > gci C:\Windows\Web\Screen | foreach { "$($_.fullname)" }
C:\Windows\Web\screen\img100.jpg
C:\Windows\Web\screen\img101.png
C:\Windows\Web\screen\img102.jpg
C:\Windows\Web\screen\img103.png
C:\Windows\Web\screen\img104.jpg
C:\Windows\Web\screen\img105.jpg
PS > gci C:\Windows\web\screen\* | foreach { "$($_.name)" }
img100.jpg
img101.png
img102.jpg
img103.png
img104.jpg
img105.jpg
An alternate (long term) solution could be to fix Get-ChildItem so that ToString() always returned the same result. This might be considered a breaking change but I suspect this "bug" has tripped up many users, even those who are already aware of it. I know I keep forgetting. Could even have Get-Item and Get-ChildItem use the same ToString(), whichever that is.
For a radical (and likely inferno inducing) solution, "fix" Get-ChildItem to only return children (or errors) and make dir a function that invoked Get-ChildItem or Get-Item as required. This would then restore the Powershell ethos of "cmdlets do what their name says they do" instead of having Get-ChildItem sometimes return children and sometimes items. (Way too much of a breaking change so not holding my breath for that one.)
Test performed using PS 5.1.18362.145.