I have the following directory tree composed of a root directory containing 10 subdirectories, and 1 file in each subdirectory.
root/
dir1/
file
dir2/
file
...
dir10/
file
I would like to edit the content of the files recursively, replacing a string "str1" by "str2". I issued the following command in powershell:
Get-ChildItem -Directory | foreach {(get-content $_/file) -replace "str1", "str2" | set-content $_/file}
And it worked like a charm, but I still do not understand how. I use a pipeline in the foreach loop, but the following call to $_ still refers to the pipeline outside the foreach loop. Why is it so?
I don't think your command did work, is the problem.
The -Directory switch of Get-ChildItem makes it only return directories, not files. If you want to return files, use the -File switch.
Next up, if you have a list of items from Get-ChildItem, those give you a System.IO.FileSystemInfo object. We can provide those directly to the Get-Content command to read the file into a string.
From a string, you can call any of the general operators PowerShell offers, including the string replace operator, -Replace. The output of this can be piped over to Set-Content and used to update or append content to an existing file.
Get-ChildItem | foreach {
(get-content $_) -replace "str1", "str2" | Set-Content $_.FullName
}
Note the only real change here is that I removed -Directory from Get-ChildItem, and then fixed the syntax on the $PSItem (the official name for the present variable in a forEach loop, often written as $_).
The reason you can use the syntax I showed is that forEach-object gives you that special $_ or $PSitem variable to use to reference $this in a collection.
The special variable $_ is not unique for the foreach loop.
It is a placeholder for the object being passed through the pipe line.
This article: POWERSHELL POWER LESSON: THIS IS NO ORDINARY VARIABLE goes into more detail about the $_ variable.
Both $_ and the | are inside the foreach loop in the curly braces { } or script block.
Related
I have multiple .txt files in folders/their sub-folders.
I want to append _old to their file names.
I tried:
Get-ChildItem -Recurse | Rename-Item -NewName {$_.name -replace '.txt','_old.txt' }
This results in:
Some files get updated correctly
Some files get updated incorrectly - they get _old twice - example: .._old_old.txt
There are few errors: Rename-Item : Source and destination path must be different.
To prevent already renamed files from accidentally reentering the file enumeration and therefore getting renamed multiple times, enclose your Get-ChildItem call in (), the grouping operator, which ensures that all output is collected first[1], before sending the results through the pipeline:
(Get-ChildItem -Recurse) |
Rename-Item -NewName { $_.name -replace '\.txt$', '_old.txt' }
Note that I've used \.txt$ as the regex[2], so as to ensure that only a literal . (\.) followed by string txt at the end ($) of the file name is matched, so as to prevent false positives (e.g., a file named Atxt.csv or even a directory named AtxtB would accidentally match your original regex).
Note: The need to collect all Get-ChildItem output first arises from how the PowerShell pipeline fundamentally works: objects are (by default) sent to the pipeline one by one, and processed by a receiving command as they're being received. This means that, without (...) around Get-ChildItem, Rename-Item starts renaming files before Get-ChildItem has finished enumerating files, which causes problems. See this answer for more information about how the PowerShell pipeline works.
Tip of the hat to Matthew for suggesting inclusion of this information.
However, I suggest optimizing your command as follows:
(Get-ChildItem -Recurse -File -Filter *.txt) |
Rename-Item -NewName { $_.BaseName + '_old' + $_.Extension }
-File limits the the output to files (doesn't also return directories).
-Filter is the fastest way to limit results to a given wildcard pattern.
$_.BaseName + '_old' + $_.Extension uses simple string concatenation via the sub-components of a file name.
An alternative is to stick with -replace:
$_.Name -replace '\.[^.]+$', '_old$&'
Note that if you wanted to run this repeatedly and needed to exclude files renamed in a previous run, add -Exclude *_old.txt to the Get-ChildItem call.
[1] Due to a change in how Get-ChildItem is implemented in PowerShell [Core] 6+ (it now internally sorts the results, which invariably requires collecting them all first), the (...) enclosure is no longer strictly necessary, but this could be considered an implementation detail, so for conceptual clarity it's better to continue to use (...).
[2] PowerShell's -replace operator operates on regexes (regular expressions); it doesn't perform literal substring searches the way that the [string] type's .Replace() method does.
The below command will return ALL files from the current folder and sub-folders within the current directory the command is executed from.
Get-ChildItem -Recurse
Because of this you are also re-turning all the files you have already updated to have the _old suffix.
What you need to do is use the -Include -Exclude paramters of the Get-Childitem Cmdlet in order to ignore files that already have the _old suffix, and meet your include criteria, for example.
Get-ChildItem -Recure -Include "*.txt" -Exclude "*_old"
Then pipe the results into your re-name item command
Get-ChildItem cmdlet explanation can be found here.
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/get-childitem?view=powershell-7
I have a list of EDI text files with specific text in them. Currently in order for our custom scripting to convert them into an SQL table, we need to be able to see the X12 file type in the filename. Because we are using SQL script to get the files into tables this solution needs to be a one line solution. We have a definition table of client files which specify which field terminator and file types to look for so we will be later substitute those values into the one line solution to be executed individually. I am currently looking at Powershell (v.3) to do this for maximum present and future compatibility. Also, I am totally new to Powershell, and have based my script generation on posts in this forum.
Files example
t.text.oxf.20170815123456.out
t.text.oxf.20170815234567.out
t.text.oxf.20170815345678.out
t.text.oxf.20170815456789.out
Search strings to find within files: (To find EDI X12 file type uniquely, which may be duplicated within the same file n times)
ST*867
ST*846
ST~867
ST~846
ST|867
ST|846
Here is what I have so far which does not show itself doing anything with the whatif parameter:
(Get-ChildItem .\ -recurse | Select-String -pattern 'ST~867' -SimpleMatch).Path | Foreach -Begin {$i=1} -Process {Rename-Item -LiteralPath $_ -NewName ($_ -replace 'out$','867.out' -f $i++) -whatif}
The fist part:
(Get-ChildItem .\ -recurse | Select-String -pattern 'ST~867' -SimpleMatch).Path
Simply gets a list of paths that we need to input to be renamed
The second part after the | pipe:
Foreach -Begin {$i=1} -Process {Rename-Item -LiteralPath $_ -NewName ($_ -replace '\.out','.867.out' -f $i++) -whatif}
will supposedly loop through that list and rename the files adding the EDI type to the end of the file. I have tried 'out$','867.out' with no change.
Current Errors:
The first part shows duplicated path elements probably because there are multiple Transaction Set Headers in the files, is there any way to force it to be unique?
The command does not show any Errors (red text) but with the whatif parameter shows that it does not rename any files (tried running it without as well).
1) remove duplicates using -List switch in Select-String
2) you need to really pipe the objects into the for loop
Try this?
Select-String -Path .\*.out -pattern 'ST~867' -SimpleMatch -List | Select-Object Path | ForEach-Object { Rename-Item $_.path ($_.path -replace 'out$','867.out') }
My problem is I want to change paths in INI Files wich are saved in a folder and its subfolders.
The path of the folder is C:\New\Path\.
Example INI file:
notAIniText = C:\A\Path\notAIniText
maybeAIniText = C:\A\Path\maybeAIniText
AIniText = C:\A\Path\AIniText
I read some other questions about PSini but I don't want to just id because I want to use the script on multiple PC and I don't want to install every time PSIni.
I tried:
$mabyIni = "C:\New\Path"
$AiniFile = Get-ChildItem - Path "C:\New\Path\*" -Include *.ini -Recurse
foreach ($file in $AiniFile) {
Select-String -Path $file -AllMatches "C:\A\Path\" | ForEach-Opject {
$file -replace [regex]:Escape('C:\A\Path'), ('$mabyIni')
} | Set-Content $mabyIni -Include *.ini
But this doesn't work. I tried it with Get-Content too, but that also doesn't work.
Is there any way whitout PSini?
The code in your comment is close, but just has a few syntax issues. It starts out strong:
$mabyIni = "C:\New\Path"
$AiniFile = Get-ChildItem - Path "C:\New\Path*" -include *.ini -recurse
ForEach($file in $AiniFile) {
So far, so good. You define the new path, and you get a list of .ini files in the old path, then you start to loop through those files. This is all good code so far. Then things start to go astray.
I see that you are trying to get the contents of each .ini file, replace the string in question, and then output that file to the new path with this:
(Get-Content $AiniFile.PSPath) | ForEach-Object {
$file -replace [regex]:Escape('C:\A\Path'),('$mabyIni')
}| Set-Content $mabyIni -include *.ini
Unfortunately you're using the wrong variables, and adding in an extra ForEach loop in there as well. Let's start with the Get-Content line. At this point in the script you are looping through files, with each current file being represented by $file. So what you really want to get the contents of is $file, and not $AiniFile.PSPath.
(Get-Content $file)
Ok, that got us the contents of that file as an array of strings. Now, I'm guessing you weren't aware, but the -Replace operator works on arrays of strings. Perfect, we just so happen to have gotten an array of strings! Since the Get-Content command is wrapped in parenthesis it completes first, we can actually just tack on the -Replace command right after it.
(Get-Content $file) -replace [regex]:Escape('C:\A\Path'),$mabyIni
Your -replace command that you had was super close! In fact, I have to give you props for using [regex]::escape() in there. That's totally a pro move, well done! The only issue with it is the replacement string didn't need to be in parenthesis, and it was single quoted, so it would not have expanded the string and your .ini files would have all had a line like:
AIniText = $mabyIni\AIniText
Not exactly what you wanted I'm guessing, so I removed the parenthesis (they weren't hurting anything, but weren't helping either, so for cleanliness and simplicity I got rid of them), and I got rid of the single quotes ' as well since we really just want the string that's stored in that variable.
So now we're looping through files, reading the contents, replacing the old path with the new path, all that's left is to output the new .ini file. It looks like they're already in place, so we just use the existing path for the file, and set the content to the updated data.
(Get-Content $file) -replace [regex]:Escape('C:\A\Path'),$mabyIni | Set-Content -Path $File.FullName
Ok, done! You just have to close the ForEach loop, and run it.
$mabyIni = "C:\New\Path"
$AiniFile = Get-ChildItem - Path "C:\New\Path*" -include *.ini -recurse
ForEach($file in $AiniFile) {
(Get-Content $file) -replace [regex]:Escape('C:\A\Path'),$mabyIni | Set-Content -Path $File.FullName
}
I am trying to compare two sets of folders to determine discrepancies in file and folder counts. I have found a command that will output the data I am looking for, but cannot find a way to print it to a file. Here is the command I am using currently:
dir -recurse | ?{ $_.PSIsContainer } | %{ Write-Host $_.FullName (dir $_.FullName | Measure-Object).Count }
This is getting me the desired data but I need to find a way to print this to a text file. Any help would be greatly appreciated.
The problem is the use of the Write-Host cmdlet, which bypasses almost all pipeline handling. In this case, it is also unnecessary, as any output that isn't used by a cmdlet is automatically passed into the pipeline (or to the console if there's nothing further).
Here is your code rewritten to output a string to the pipeline instead of using Write-Host. This uses PowerShell's string subexpression operator $(). At the console, it will look the same, but it can be piped to a file or other cmdlet.
gci -Recurse -Directory | %{ "$($_.FullName) $((gci $_.FullName).Count)" }
You may also find it useful to put the data into a PSCustomObject. Once you have the object, you can do further processing such as sorting or filtering based on the count.
$folders = gci -Recurse -Directory | %{ [PSCustomObject]#{Name=$_.FullName; Count=(dir $_.FullName).Count }}
$folders | sort Count
$folders | where Count -ne 0
Some notes on idioms: dir is an alias for Get-Childitem, as is gci. Using gci's -Directory parameter is the best way to list only directories, rather than the PSIsContainer check. Finally, Measure-Object is unnecessary. You can take the Count of the file listing directly.
See also Write-Host Considered Harmful from the inventor of PowerShell
I have a problem getting a filter argument to Get-ChildItem in a function.
The following works fine and displays a whole list of files:
c:\temp\Get-ChildItem -Include *deleteme*.txt -Recurse
Now say I have the following script
#file starts here
#filename = GetLastFile.ps1
param([string] $filter)
$files = Get-ChildItem $filter
Write-Host $files #should print all matching files but prints nothing
$file = $files | Select-Object -Last 1;
$file.name #returns filename
#File ends here
Now trying to run the script,
c:\temp.\GetLastFile.ps1 "-Include *deleteme*.txt -Recurse"
returns nothing.
Supplying a filter, *.*, works fine. It seems to be failing due to the -Include or -Exclude. Any ideas?
You're starting to get into an area where where Powershell 2.0 proxy functions can help. However, short of that, here's a simple way in PowerShell 2.0 to do this assuming all you need is -Include and -Recurse. Actually, I would recommend using -Filter instead it will do what you want and frankly it's quite a bit faster (4x on some of my tests) because -filter uses filesystem filtering provided by the OS whereas -include is processed by PowerShell.
param([string]$Filter, [switch]$Recurse)
$files = Get-ChildItem #PSBoundParameters
Write-Host $files #should print all matching files but prints nothing
$file = $files | Select-Object -Last 1;
$file.name #returns filename
The # symbol is used to "splat" an array or hashtable across the parameters to a command. The $PSBoundParameters variable is an automatic variable new to PowerShell 2.0 that is defined in functions. It's a hashtable that contains all the bounded (named and positional) parameters e.g.:
PS> function foo($Name,$LName,[switch]$Recurse) { $PSBoundParameters }
PS> foo -Name Keith Hill -Recurse
Key Value
--- -----
Name Keith
Recurse True
LName Hill
When you splat a hashtable like this against a command, PowerShell will map the key's (e.g. Recurse) value to the parameter named Recurse on the command.
I believe what is happening is your $filter parameter is being treated as a single string argument to the Get-ChildItem command. As such, unless you have a directory named "-Include deleteme.txt -Recurse", the command will always return nothing.
As for fixing your problem, well, there are a bunch of ways you could approach it. Probably one of the more versatile ways is to switch program behavior if the the $filter argument is passed, and instead of passing the entire filter string just pass the "deleteme.txt" string.
You can use Invoke-Expression to execute a command stored in a variable. For example:
param([string] $filter)
$files = Invoke-Expression "Get-ChildItem $filter"
Write-Host $files
$file = $files | Select-Object -Last 1
$file.name