Powershell add text to files if does not exist - powershell

I would like to create a Powershell script that adds text to *.jsp *.sh *.cmd files. I would also like it to check if the text already exist in that file and if it does then it skips the file. So far I have found Select-String which will find the text in the file. How do I then use this information to add text to top of the files that do not have the text in them? I also found Add-Content which seems to add the content I want but I would like to do it at the beginning and have some logic to not just keep re-adding it every time I run the ps1.
Select-String -Pattern "TextThatIsInMyFile" -Path c:\Stuff\batch\*.txt
Add-Content -Path "c:\Stuff\batch\*.txt" -Value "`r`nThis is the last line"

Very similar to what #MikeWise has, but a little better optimized. I have it pulling the data and making the provider filter the files returned (much better than filtering afterwards). Then I pass it to a Where statement using Select-String's -quiet parameter to provide boolean $true/$false to the Where. That way only file that you want to look at are even looked at, and only those missing the text you need are altered.
Get-ChildItem "C:\Stuff\Batch\*" -Include *.jsp,*.sh,*.cmd -File |
Where{!(Select-String -SimpleMatch "TextThatIsInMyFile" -Path $_.fullname -Quiet)} |
ForEach{
$Path = $_.FullName
"TextThatIsInMyFile",(Get-Content $Path)|Set-Content $Path
}
Edit: As you discovered the \* does not work with -Recursive. Use the following if you need to recurse:
Get-ChildItem "C:\Stuff\Batch" -Include *.jsp,*.sh,*.cmd -File -Recurse

Refering to the manual: https://technet.microsoft.com/en-us/library/hh849903.aspx
Specifically:
Outputs
The output type is the type of the objects that the cmdlet emits.
• Microsoft.PowerShell.Commands.MatchInfo or System.Boolean
By default, the output is a set of MatchInfo objects, one for each match found.
If you use the Quiet parameter, the output is a Boolean value indicating whether the pattern was found.
I also think that you also need to do this per file. So (presumbably):
$files = Get-ChildItem "C:\Stuff\batch" -Filter *.txt
for ($i=0; $i -lt $files.Count; $i++)
{
$filename = $files[$i].FullName
$b = Select-String -Quiet -Pattern "TextThatIsInMyFile" -Path $fileName
if (-not $b)
{
Add-Content -Path $fileName -Value "`r`nTextThatIsInMyFile"
}
}
I tested this and I think it does what you want, i.e. adds the text to the end of files that do not have it, and not doing it more than once.

Related

In Powershell how do I find file names that contains text or files that have the text in them

I'm searching through directories recursively. I want to find files that contain text i'm looking for OR the text is in the content of the file.
For example, if I search for "hello", i'd normally do this:
Find matching file names:
get-childitem -filter "*hello*"
Find files that have text in them:
get-childitem -recurse | select-string -pattern "*hello*"
But I want to do both at the same time. Which means you could have files that don't have "hello" in the name but it does appear in the contents of the file. Or visa versa.
EDIT: I considered using where-object with an -or but having trouble figuring out how to construct that properly.
EDIT: My mistake, meant to include select-string in the example.
Ideas?
Thanks
I don't think its possible to use -Filter because you might be excluding those files which's content could contain the word you're looking for.
An easy approach I could think of, is looping through all files in $startPath recursively and, if the word is contained in the file's name, go to the next iteration with continue or break if you want to stop the loop at first finding, and of course, use -Raw for Get-Content:
$startPath = 'C:\path\to\startFolder'
$magicWord = 'hello'
foreach($file in Get-ChildItem $startPath -Recurse -File)
{
if($file.Name -match $magicWord)
{
$file
continue
# or break here if you want to stop the loop
}
if((Get-Content $file.FullName -Raw) -match $magicWord)
{
$file
# break here if you want to stop the loop
}
}
Not sure if using this would be faster or not:
if([system.io.file]::ReadAllText($file.FullName) -match $magicWord)
{
...
}

How would I specify a directory to run a PowerShell script that would edit file extensions?

I am new to PowerShell and new to IT. I've been asked by my boss to write a PowerShell script that will identify filenames that have no file extension and then change them to .PDF files. After doing some research online I've found a script that had a similar purpose and tried to tailor it to my needs:
$proj_files = Get-ChildItem | Where-Object {$_.Extension -eq "."}
ForEach ($file in $proj_files) {
$filenew = $file.Name + ".pdf"
Rename-Item $file $filenew
}
My first question is does the logic in this script make sense? Is "Extension -eq "." the correct syntax to specify a filename with no extension? My other thought was to use Extension -eq "null" or something similar. If I do need to use a null value, what would that look like? My other question is how would I specify a given directory for this script to search through, or would I even need to? My thought here would be to specify the path under Get-ChildItem, like so: $proj_files = Get-ChildItem -Path C:\Users\mthomas\Documents | Where-Object {$_.Extension -eq ".'} Does that seem correct? I am hesitant to test this out before getting a second opinion because I don't want to change every file extension on my computer or something stupid like that. Anyhow, thanks everyone for the help.
You can do something like the following to find files in a directory without an extension, and rename them to have a PDF extension:
$directory = "C:\Path\To\Directory"
Get-ChildItem -File $directory | Where-Object { -Not $_.Extension } | Foreach-Object {
$_ | Rename-Item -NewName "$($_.Name).pdf"
}
Let's break this down
$directory = "C:\Path\To\Directory"
This is where we set the directory we want to locate files without extensions in. It doesn't have to be set as a static variable but since you are just getting your feet wet with Powershell this keeps it simple.
Get-ChildItem -File $directory
Get-ChildItem is the cmdlet which is used to list directory contents (also aliased to gci, ls, and dir). -File tells it to only list files, and $directory references the directory we want to search from, which we set above. Note that Get-ChildItem might behave differently depending on the provider (for example, you can also use Get-ChildItem on a registry key), but if you are working with a filesystem path you do not need to worry about additional providers for this case.
|
Passes the previous output down the pipeline. This is a common operator in Powershell, but basically you can string commands together using it. You can read more about the pipeline at https://learn.microsoft.com/en-us/powershell/scripting/getting-started/fundamental/understanding-the-windows-powershell-pipeline?view=powershell-6
Where-Object { -Not $_.Extension }
Where-Object evaluates a condition on one or more items, and filters out items that do not meet the condition. Since Get-ChildItem can return one or more files, we use the -Not operator in the ScriptBlock (denoted by {} and make sure that there is no extension on the file. $_, or $PSItem, is a special variable used by the pipeline, in this case $_ equals each item returned by Get-ChildItem. The Extension property exists on files returned by Get-ChildItem, and will be blank, or evaluated as $False. So filtering on -Not $_.Extension is the same as saying to only match objects that are missing a file extension. Where-Object can be read about in more detail here: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/where-object?view=powershell-6
Foreach-Object { SCRIPTBLOCK }
Similar to Where-Object, but runs code for each object in the pipeline rather than evaluating and filtering out objects which don't match a condition. In this case, we pipe the each file without an extension to Rename-Item, which I'll break down further below. More information on Foreach-Object can be read about here: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object?view=powershell-6
$_ | Rename-Item -NewName "$($_.Name).pdf"
Rename the current file in the Foreach-Object block to the new name with .pdf appended. The "$( ... )" is called a sub-expression, which is a string interpolation technique that lets you run a command within a string, and make its output part of the string. You could achieve the same effect by doing $_ | Rename-Item -NewName ( $_.Name + ".pdf" ) which just adds a .pdf to the end of the current name.
Summary
The pipeline is a very powerful tool in Powershell, and is key to writing efficient and less bloated scripts. It might seem complex at first, but the more you use it the less daunting it will seem. I highly suggest reading the additional documentation I linked to above as it should help fill in any gaps I may have missed in my explanations above.
To simplify the breakdown above, the command does this, in this order: Gets all files in the specified directory, selects only the files that do not have an extension, then renames each file found without an extension to have a .pdf at the end.
The logic in the script - the overall shape - makes understandable sense, but is not right for it to work as you intend.
Testing on my computer here:
new-item -ItemType File -Name 'test'
get-item test | format-list *
get-item test | foreach { $_.extension; $_.Extension.length; $_.extension.GetType().name }
a file with no extension shows up with an empty string (blank content, length 0, type String, so your where-object { $_.Extension -eq "." } needs to be looking for "" instead of ".".
But:
Get-ChildItem | Where-Object { $_.Extension -eq '' }
shows me some folders as well, because they also have no extension in their name, so you might want Get-ChildItem -File to restrict it to just files.
how would I specify a given directory for this script to search through, or would I even need to?
It would run in the current directory, whichever shows up in your prompt PS C:\wherever> so if you need it to run somewhere else, yes you'd need to change to that folder or specify in get-childitem -LiteralPath 'c:\path\to\wherever'. You haven't mentioned subfolders, if you need those included, get-childitem -Recurse switch as well.
Speaking of subfolders, your $filenew = $file.Name + ".pdf" only makes sense in the current directory, I think it would work better if you used the full filename including path, so they definitely get renamed in the same place they were found $filenew = $file.FullName + ".pdf"
Is "Extension -eq "." the correct syntax to specify a filename with no extension?
Being careful here, what you wrote in your question was correct syntax but incorrect string content. What you've written here with quotes on the left of Extension is incorrect syntax.
My other thought was to use Extension -eq "null" or something similar. If I do need to use a null value, what would that look like?
And being careful here, "null" is not a null value, it's a string containing the four letter word 'null'.
You don't need to use a null value here, normally if you do it looks like $null, but in this case you could use where-object { [string]::IsNullOrEmpty($_.Extension) } but there's no benefit to it, I think.
And, as a stylistic choice, both "" and '' are strings, but "" can contain variables and sub-expressions, so if you have plain text it's a neat habit to use '' for it because it makes it clear to the reader that you intend there to be nothing special happening in this string.
Then your code, with parameter names given, looks more like:
$proj_files = Get-ChildItem -LiteralPath 'C:\Users\mthomas\Documents' |
Where-Object {$_.Extension -eq '.'}
foreach ($file in $proj_files)
{
$filenew = $file.FullName + '.pdf'
Rename-Item -LiteralPath $file.FullName -NewName $filenew
}
If you want to see what it will do, use -WhatIf on the end of Rename-Item:
Rename-Item -LiteralPath $file.FullName -NewName $filenew -WhatIf
Then it won't make the changes, just tell you what it would do.
I am hesitant to test this out before getting a second opinion because I don't want to change every file extension on my computer or something stupid like that
Sensible. But internet people are going to tell you to test their code before running it, because ultimately it's your responsibility to safeguard your files, rather than trust random code from the internet, so having test folders, having a spare machine, having a good backup, playing with PowerShell in pieces until you are happy with what they do, they're all good habits to get into as well.

Powershell - Match ID's in a text file against filenames in multiple folders

I need to search through 350,000 files to find any that contains certain patterns in the filename. However, the list of patterns (id numbers) that it needs to match is 1000! So I would very much like to be able to script this, because they were originally planning on doing it manually...
So to make it clearer:
Check each File in folder and all subfolders.
If the filename contains any of the IDs in the text file then move it to another file
Otherwise, ignore it.
So I have the basic code that works with a single value:
$name = Get-Content 'C:\test\list.txt'
get-childitem -Recurse -path "c:\test\source\" -filter "*$name*" |
move-item -Destination "C:\test\Destination"
If I change $name to point to a single ID, it works, if I have a single ID in the txt file, it works. Multiple items in a list:
1111111
2222222
3333333
It fails. What am I doing wrong? How can I get it to work? I'm still new to powershell so please be a little more descriptive in any answers.
Your test fails because it is effectively trying to do this (using your test data).
Get-ChildItem -Recurse -Path "c:\test\source\" -filter "*1111111 2222222 3333333*"
Which obviously does not work. It is squishing the array into one single space delimited string. You have to account for the multiple id logic in a different way.
I am not sure which of these will perform better so make sure you test both of these with your own data to get a better idea of execution time.
Cycle each "filter"
$filters = Get-Content 'C:\test\list.txt'
# Get the files once
$files = Get-ChildItem -Recurse -Path "c:\test\source" -File
# Cycle Each ID filter manually
$filters | ForEach-Object{
$singleFilter
$files | Where-Object{$_.Name -like "*$singleFilter*"}
} | Move-Item -Destination "C:\test\Destination"
Make one larger filter
$filters = Get-Content 'C:\test\list.txt'
# Build a large regex alternative match pattern. Escape each ID in case there are regex metacharacters.
$regex = ($filters | ForEach-Object{[regex]::Escape($_)}) -join "|"
# Get the files once
Get-ChildItem -Recurse -path "c:\test\source" -File |
Where-Object{$_.Name -match $regex} |
Move-Item -Destination "C:\test\Destination"
try following this tutorial on how to use get-content function. Looks like when you have a multiple line file, you get an array back. you then have to iterate through your array and use the logic you used for only one item

powershell slow(?) - write names of subfolders to a text file

My Powershell script seems slow, when I run the below code in ISE, it keeps running, doesn't stop.
I am trying to write the list of subfolders in a folder(the folder path is in $scratchpart) to a text file. There are >30k subfolders
$limit = (Get-Date).AddDays(-15)
$path = "E:\Data\PathToScratch.txt"
$scratchpath = Get-Content $path -TotalCount 1
Get-ChildItem -Path $scratchpath -Recurse -Force | Where-Object { $_.PSIsContainer -and $_.CreationTime -lt $limit } | Add-Content C:\Data\eProposal\POC\ScratchContents.txt
Let me know if my approach is not optimal. Ultimately, I will read the text file, zip the subfolders for archival and delete them.
Thanks for your help in advance. I am new to PS, watched few videos on MVA
Add-Content, Set-Content, and even Out-File are notoriously slow in PowerShell. This is because each call opens the file, writes to it, and closes the handle. It never does anything more intelligently than that.
That doesn't sound bad until you consider how pipelines work with Get-ChildItem (and Where-Object and Select-Object). It doesn't wait until it's completed before it begins passing objects into the pipeline. It starts passes objects as soon as the provider returns them. For a large result set, this means that the objects are still feeding in the pipeline long after several have finished processing. Generally speaking, this is great! It means the system will function more efficiently, and it's why stuff like this:
$x = Get-ChildItem;
$x | ForEach-Object { [...] };
Is significantly slower than stuff like this:
Get-ChildItem | ForEach-Object { [...] };
And it's why stuff like this appears to stall:
Get-ChildItem | Sort-Object Name | ForEach-Object { [...] };
The Sort-Object cmdlet needs to waits until it's received all pipeline objects before it sorts. It kind of has to to be able to sort. The sort itself is nearly instantaneous; it's just the cmdlet waiting until it has the full results.
The issue with Add-Content is that, well, it experiences the pipeline not as, "Here's a giant string to write once," but instead as, "Here's a string to write. Here's a string to write. Here's a string to write. Here's a string to write." You'll be sending content to Add-Content here line by line. Each line will instantiate a new call to Add-Content, requiring the file to open, write, and close. You'll likely see better performance if you assign the result of Get-ChildItem [...] | Where-Object [...] to a variable, and then write the entire variable to the file at once:
$limit = (Get-Date).AddDays(-15);
$path = "E:\Data\PathToScratch.txt";
$scratchpath = Get-Content $path -TotalCount 1;
$Results = Get-ChildItem -Path $scratchpath -Recurse -Force -Directory | `
Where-Object{$_.CreationTime -lt $limit } | `
Select-Object -ExpandPropery FullName;
Add-Content C:\Data\eProposal\POC\ScratchContents.txt -Value $Results;
However, you might be concerned about memory usage if your results are actually going to be extremely large. You can actually use System.IO.StreamWriter for this purpose, too. My process improved in speed by nearly two orders of magnitude (from 12 hours to 20 minutes) by switching to StreamWriter and also only calling StreamWriter when I had about 250 lines to write (that seemed to be the break-even point for StreamWriter's overhead). But I was parsing all ACLs for user home and group shares for about 10,000 users and nearly 10 TB of data. Your task might not be as large.
Here's a good blog explaining the issue.
Do you have at least PowerShell 3.0? If you do you should be able to reduce the time by filtering out the files since you are returning those as well.
Get-ChildItem -Path $scratchpath -Recurse -Force -Directory | ...
Currently you are returning all files and folders then filtering out the files with $_.PSIsContainer which would be slower. So should end up with something like this
Get-ChildItem -Path $scratchpath -Recurse -Force -Directory |
Where-Object{$_.CreationTime -lt $limit } |
Select-Object -ExpandPropery FullName |
Add-Content C:\Data\eProposal\POC\ScratchContents.txt

Replace text between quotes in XML

I'm trying to find a way to change unknown text (could be anything) in an XML file for a printer migration.
The text that is in question is
PrintProcessor="hpcpp111"
The Print Processor section could contain anything as this varies depending on printer model, type and driver. I would prefer to use powershell if possible as I am trying to understand scripting and how it all works, but I find it a little confusing. I may need to manually edit thousands of these files as we are migrating 5,500 printers.
I have found some code that gets it close for example
function Reset-InfoPathTemplateLink {
Param(
[string]$FilePath,
[string]$FileExtension,
[string]$OldPath,
[string]$NewPath
)
$files = Get-ChildItem $FilePath -Filter $FileExtension
foreach ($file in $files) {
(Get-Content $file.fullname) |
ForEach-Object {$_ -replace $OldPath,$NewPath} |
Set-Content $file.fullname
} #end foreach
} #end function
and if I dot-source it and then run this command
Reset-InfoPathTemplateLink -FilePath "c:\test2" -FileExtension ".xml" -OldPath "PrintProcessor=""" -NewPath "PrintProcessor='"WinPrint"'"
The WinPrint gets added to the beginning of the current print processor, which isnt ideal. If anyone has tips on how to get this to work, or if they can suggest a better method of doing this I would appreciate it
this is how you can use PowerShell to replace a pattern within a text file:
$path="c:\myPath.txt"
(Get-Content $path) -replace '(PrintProcessor=")(.*)(")','$1WinPrint$3' | Set-Content $path