Powershell script for purging files as per creation year - powershell

I wrote a powershell script which will iterate through three different path and get list of files that are less then 7 years and delete them from current timestamp.
I am getting creation year of file and if am able to recursively iterate through all those three path.
Problem is out of 3, two paths have too many folders and files due to which when script is in loop it shows memory exception. Also I will not be able to set maxmemorypershellMB, since I don't have access.
Anything else that I can do this to avoid memory exception
this is piece of code below:
$files = Get-ChildItem "$path" –Recurse -file
for ($i=0; $i -lt $files.Count; $i++) {
$outfile = $files[$i].FullName #file name
$FileDate = (Get-ChildItem $outfile).CreationTime #get creation date of file
$creationYear = $FileDate.Year
$creationMonth =$FileDate.Month #get only year out of creation date
If( $creationYear -lt $purgeYear ){
If (Test-Path $outfile){ #check if file exist then only proceed
$text=[string]$creationYear+" "+$outfile
$text >> 'listOfFilesToBeDeleted_PROD.txt' #this will get list of files to be deleted
#remove-item $outfile
}
}
}

You could try to filter the the files using where-object instead of a for loop:
$limit = (Get-Date).AddYears(-7)
$path = "c:\"
$outfile = "c:\test.txt"
Get-ChildItem -Path "$path" -Recurse -file |
Where-Object { $_.CreationTime -lt $limit } |
foreach { '{0} {1}' -f $_.CreationTime, $_.FullName |
Out-File -FilePath $outfile -Append }
Solution for your comment:
# retrieve all affected files and select the fullname and the creationtime
$affectedFiles = Get-ChildItem -Path "$path" -Recurse -file |
Where-Object { $_.CreationTime.Year -lt $purgeYear } |
select FullName, CreationTime
foreach ($file in $affectedFiles)
{
# write the file to listOfFilesToBeDeleted
'{0} {1}' -f $file.CreationTime.Year, $file.FullName |
Out-File -FilePath listOfFilesToBeDeleted.txt -Append
# delete the file
Remove-Item -Path $file.FullName -Force
}

Related

Find similarly-named files, and if present, remove the files without a specific string using PowerShell

In a directory, there are files with the following filenames:
ExampleFile.mp3
ExampleFile_pn.mp3
ExampleFile2.mp3
ExampleFile2_pn.mp3
ExampleFile3.mp3
I want to iterate through the directory, and IF there is a filename that contains the string '_pn.mp3', I want to test if there is a similarly named file without the '_pn.mp3' in the same directory. If that file exists, I want to remove it.
In the above example, I'd want to remove:
ExampleFile.mp3
ExampleFile2.mp3
and I'd want to keep ExampleFile3.mp3
Here's what I have so far:
$pattern = "_pn.mp3"
$files = Get-ChildItem -Path '$path' | Where-Object {! $_.PSIsContainer}
Foreach ($file in $files) {
If($file.Name -match $pattern){
# filename with _pn.mp3 exists
Write-Host $file.Name
# search in the current directory for the same filename without _pn
<# If(Test-Path $currentdir $filename without _pn.mp3) {
Remove-Item -Force}
#>
}
enter code here
You could use Group-Object to group all files by their BaseName (with the pattern removed), and then loop over the groups where there are more than one file. The result of grouping the files and filtering by count would look like this:
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1
Count Name Group
----- ---- -----
2 ExampleFile {ExampleFile.mp3, ExampleFile_pn.mp3}
2 ExampleFile2 {ExampleFile2.mp3, ExampleFile2_pn.mp3}
Then if we loop over these groups we can search for the files that do not end with the $pattern:
#'
ExampleFile.mp3
ExampleFile_pn.mp3
ExampleFile2.mp3
ExampleFile2_pn.mp3
ExampleFile3.mp3
'# -split '\r?\n' -as [System.IO.FileInfo[]] | Set-Variable files
$pattern = "_pn"
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1 | ForEach-Object {
$_.Group.Where({-not $_.BaseName.Endswith($pattern)})
}
This is how your code would look like, remove the -WhatIf switch if you consider the code is doing what you wanted.
$pattern = "_pn.mp3"
$files = Get-ChildItem -Path -Filter *.mp3 -File
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1 | ForEach-Object {
$toRemove = $_.Group.Where({-not $_.BaseName.Endswith($pattern)})
Remove-Item $toRemove -WhatIf
}
I think you can get by here by adding file names into a hash map as you go. If you encounter a file with the ending you are interested in, check if a similar file name was added. If so, remove both the file and the similar match.
$ending = "_pn.mp3"
$files = Get-ChildItem -Path $path -File | Where-Object { ! $_.PSIsContainer }
$hash = #{}
Foreach ($file in $files) {
# Check if file has an ending we are interested in
If ($file.Name.EndsWith($ending)) {
$similar = $file.Name.Split($ending)[0] + ".mp3"
# Check if we have seen the similar file in the hashmap
If ($hash.Contains($similar)) {
Write-Host $file.Name
Write-Host $similar
Remove-Item -Force $file
Remove-Item -Force $hash[$similar]
# Remove similar from hashmap as it is removed and no longer of interest
$hash.Remove($similar)
}
}
else {
# Add entry for file name and reference to the file
$hash.Add($file.Name, $file)
}
}
Just get a list of the files with the _pn then process against the rest.
$pattern = "*_pn.mp3"
$files = Get-ChildItem -Path "$path" -File -filter "$pattern"
Foreach ($file in $files) {
$TestFN = $file.name -replace("_pn","")
If (Test-Path -Path $(Join-Path -Path $Path -ChildPath $TestFN)) {
$file | Remove-Item -force
}
} #End Foreach

Copy-Item by header & timestamp

I am using the code below to filter out files depending on the headers in the file.
It works like a charm, but I have a problem with that it takes all the files in the $InputDirectory.
I would like to limit it so it only takes files that are 1-2 hours old.
There are two ways where I can get the date for this process.
Filename contains timestamp = XXXXXXXXXXX_XXXXXXXX_valuereport_YYYYMMDDhhmmss.csv
The timestamp the file was created (please note we are talking about 800K-1M files in the directory and more is added every hour, so the fastest way would be appreciated.
So how do I insert something in my code, so it besides the header, only takes files that are <1-2 hours old?
Sorry about the code example, I am new to this site and not sure how to get it in the right order.
Nothing yet.
foreach ($FilePath in (Get-ChildItem $InputDirectory -File) | Select-Object -ExpandProperty FullName) {
$Header = Get-Content $FilePath -First 1
# test for a string in the header line that distincts it from the other files
if ($Header -match ';energy,Wh,') {
# the substring ';energy,Wh,' defines this file as a 'HeatMeter' file
Copy-Item -Path $FilePath -Destination $OutputPathHeat
} elseif ($Header -match ';fabrication-no,,inst-value,0,0,0;datetime,,inst-value,0,0,0;volume,m3') {
# the substring ';datetime,,inst-value,0,0,0;volume,m3' defines this file as a 'WaterMeter' file
Copy-Item -Path $FilePath -Destination $OutputPathWater
} else {
# if all key substrings above did not match, move to the 'Other' directory
Copy-Item -Path $FilePath -Destination $OutputPathOther
}
There are several ways to filter a directory listing. The easiest way is to pipe the result of Get-ChildItem through Where-Object like:
Get-ChildItem -Path $InputDirectory -File |
Where-Object { $_.CreationTime -gt (Get-Date).AddHours(-2) } |
Select-Object -ExpandProperty FullName |
ForEach-Object {
$FilePath = $_
$Header = Get-Content $FilePath -First 1
# test for a string in the header line that distincts it from the other files
if ($Header -match ';energy,Wh,') {
# the substring ';energy,Wh,' defines this file as a 'HeatMeter' file
Copy-Item -Path $FilePath -Destination $OutputPathHeat
}
elseif ($Header -match ';fabrication-no,,inst-value,0,0,0;datetime,,inst-value,0,0,0;volume,m3') {
# the substring ';datetime,,inst-value,0,0,0;volume,m3' defines this file as a 'WaterMeter' file
Copy-Item -Path $FilePath -Destination $OutputPathWater
}
else {
# if all key substrings above did not match, move to the 'Other' directory
Copy-Item -Path $FilePath -Destination $OutputPathOther
}
}
It checks that the CreationTime is greater than now - 2h. Note that the last modified (LastWriteTime) timestamp may also be suitable for your use case.

How to add headers to csv file

I am creating a script to get the file version of the dlls and export the output to a csv file , but I want to give particular headers to csv file. How should I do it ? I also need to add a new system date and time column to the csv file.
$j = 'C:\Program Files (x86)\anyfilepatha\'
$files = get-childitem $j -recurse -Include *.dll
$cvsdataFile = 'C:\Program Files\MySQL\Connector ODBC 8.0\dllsinfo.csv'
# Add-Content -Path "C:\Program Files\MySQL\Connector ODBC 8.0\dllsinfo.csv" -Value "Dlls" , "Version Name" , "Location"
$header = foreach ($i in $files)
{
if($i.Name -like '*Eclipsys*' -or $i.Name -like '*Helios*')
{
continue;
}
$verison = [System.Diagnostics.FileVersionInfo]::GetVersionInfo($i).FileVersion
if($verison -eq $null)
{
$i.FullName | Out-File NoVersion.txt -append
}
else
{
Write-Host $i ----> $verison
"{0}`t{1}`t{2} " -f $i, [System.Diagnostics.FileVersionInfo]:: GetVersionInfo($i).FileVersion, $i.CreationTime, $i.FullName | Out-File -Append $cvsdataFile
}
}
You could do it with a one liner. You can exclude items with -Exclude parameter and PowerShell calculated properties for adding new columns. See more about calculated properties here.
Get-ChildItem -Path $j -File -Exclude '*Eclipsys*','*Helios*' -Include *.dll |
Select-Object -Property #{E={$_.versioninfo.Fileversion};l='Version'},#{E={Get-Date -f 'dd/mm/yyyy'};l='Date'},#{E={Get-Date -f 'hh:mm:ss ttt='};L='Time'}

PS - How can I remove a file name entry from a text file

I get a list of extracted files without their full path. They come in a txt file on seperate lines. These files can be found in different folders or subfolders. I'd like to delete the files from the list and remove the file name from the list only if they're older than 30 minutes.
list.txt example:
file1.doc
file2.doc
file3.doc
Let's say file3.doc is less than 30 minutes old. With my current code I can get as far as deleting file1.doc and file2.doc and not touch file3.doc no problem. I would like my code to remove file1.doc and file2.doc from list.txt as it deletes the files.
$Now = Get-Date
$Minutes = "30"
$TargetFolder = "C:\Test"
$LastWrite = $Now.AddMinutes(-$Minutes)
$Files = Get-Content C:\list.txt |% {get-childitem $TargetFolder -include $_ -recurse} | Where {$_.LastWriteTime -le "$LastWrite"}
foreach ($File in $Files)
{
if ($File -ne $NULL)
{
Remove-Item $File.FullName -WhatIf | out-null
}
}
I am a total beginner How would I go about removing the file name from the original list C:\list.txt?
Thank you.
I got it using Set-Content in the following manner. Feel free to comment if you think i'm doing something wrong, but this works me very well as it is.
$Now = Get-Date
$Minutes = "30"
$TargetFolder = $args[0]
$LastWrite = $Now.AddMinutes(-$Minutes)
$List = "C:\list.txt"
$Files = Get-Content $List |% {get-childitem $TargetFolder -include $_ -recurse} | Where {$_.CreationTime -le "$LastWrite"}
foreach ($File in $Files)
{
if ($File -ne $NULL)
{
Add-Content -path C:\deleted.txt -value ((get-date).ToShortDateString() + ' - ' + (get-date).ToShortTimeString() + ' ' + $File.FullName)
Remove-Item $File.FullName | out-null
Set-Content -Path $List -Value (get-content -Path $List | Select-String -Pattern $File.Name -NotMatch)
}
}

Write folder name and subfolder name being deleted to output file

I have written the below with the intention of deleting all folders in a directory that have a creation date of 2 days or more and log this in an output file if it is successful or not.
The script works as I would like with the exception that the name of the file will not show in the output file. All that is displayed is 'Deletion of Failed/Successful'
$dump_path = "C:\desktop"
$max_days = "-2"
$curr_date = Get-Date
$del_date = $curr_date.AddDays($max_days)
ForEach-Object {
$filename = $_
Get-ChildItem $statfolder\$_ -Recurse | Where-Object {
$_.CreationTime -lt $del_date
} | Remove-Item -Recurse -Force
if ($? -eq $false) {
echo "$Deletion of $filename Failed" |
Out-File -Append C:\Logs\DELETION_FAIL_K_$(Get-Date -Format `"dd-MMM-yyyy`").txt
} else {
Write-Output "Deletion of $filename Successful" |
Out-File -Append C:\Logs\DELETION_SUCCESS_K_$(Get-Date -format `"dd-MMM-yyyy`").txt
}
}
I would ideally like the log to display the parent folder name and a list of the sub folders in next level down only. Is this possible?
Eg. the log would read
Deletion of folder 12-Jan-2017 containing sub folders R2015, R2086 was Successful
If the sub folders in the next level cannot be added then just the below would be great:
Deletion of folder 12-Jan-2017 was Successful
The way you're using ForEach-Object the current object variable $_ (and consequentially the variable $filename) is never populated. Where would you expect the value to come from?
Feed the output of Get-ChildItem | Where-Object into ForEach-Object, but sort the results by full name first, so that nested folders are deleted before their parents.
Get-ChildItem $statfolder\$_ -Recurse | Where-Object {
$_.PSIsContainer -and
$_.CreationTime -lt $del_date
} | Sort-Object FullName | ForEach-Object {
$folder = $_.FullName
"Deleting folder '$folder'."
Remove-Item $folder -Recurse -Force -WhatIf
}
With the -WhatIf switch present you'll be doing a dry-run, just echoing what would be deleted without actually deleting it. After you verified that everything would work as intended remove the switch and re-run.