I have few folders in which my script runs and triggers job for each folder. Imagine if I have 10 subfolders inside the main folder and I want to limit the number of jobs running to at a time to be 5. how can I do that? I will put my code below.
function ThrottleJob([int]$MaxJob) {
do {
$jobs = Get-Job -state "Running"
Start-Sleep -Seconds 1
} while ($jobs.Count -gt $MaxJob)
}
$folders = #()
$folders += (Get-Item $path).FullName
$folders += Get-ChildItem -Path $path -Recurse -Directory
foreach ($folder in $folders) {
if (Get-ChildItem -Path $folder) {
ThrottleJob -MaxJob 5
Start-Job -InitializationScript $allJobs -ScriptBlock { runJob -path $args[0]} -ArgumentList $folder | Out-Null
}
}
Could anyone please help me out?
Here you can an example for setting a threshold for Start-Job, I do believe a normal foreach loop will be faster than this.
using namespace System.Collections.Generic
using namespace System.Management.Automation
$maxJobs = 5
$path = "$HOME\Documents"
$folders = #(
(Get-Item $path).FullName
(Get-ChildItem -Path $path -Recurse -Directory).FullName
)
$jobs = [List[Job]]::new()
foreach($folder in $folders)
{
$job = Start-Job {
Get-ChildItem $using:folder -File | Select-Object DirectoryName, Name
}
$jobs.Add($job)
while(($jobs.State -eq 'Running').Count -ge $maxJobs)
{
Clear-Host
Get-Job -State Running | Format-Table -AutoSize
Start-Sleep -Seconds 1
}
}
$result = Receive-Job $jobs -Wait -AutoRemoveJob
$result | Format-Table
As you can see, the code is awfully complicated. This is how it would look using Start-ThreadJob, which will most likely be faster than a foreach loop and will consume less memory than Start-Job.
$maxJobs = 5
$path = "$HOME\Documents"
$folders = #(
(Get-Item $path).FullName
(Get-ChildItem -Path $path -Recurse -Directory).FullName
)
$jobs = foreach($folder in $folders)
{
Start-ThreadJob {
Get-ChildItem $using:folder -File | Select-Object DirectoryName, Name
} -ThrottleLimit $maxJobs
}
$result = Receive-Job $jobs -Wait -AutoRemoveJob
$result | Format-Table
I am writing an application that looks through a directory tree and reports if a folder is inactive based on last write time and read only attribute.
However my loop stops after like 7 iterations even though there are thousands of folders.
My code looks like:
function FolderInactive{
Param([string]$Path)
$date = (Get-Date).AddDays(-365)
$anyReadOnly = $false
Get-ChildItem $Path -File -ErrorAction SilentlyContinue | ForEach-Object {
if($_.LastWriteTime -ge $date){
$false
continue
}
if($_.IsReadOnly -eq $false){
$anyReadOnly = $true
}
}
$anyReadOnly
}
Get-ChildItem "some drive" -Recurse | where {$_.PSIsContainer} | Foreach-Object {
Write-Host $_.FullName
FolderInactive($_.FullName)
}
If I comment out the FolderInactive function call in the Foreach loop it prints all the folders, but with the function call it stops after a few iterations. What is happening?
You cannot use continue with the Foreach-Object cmdlet. Foreach-Object is a cmdlet, not a loop. You instead want to use the loop:
function FolderInactive{
Param([string]$Path)
$date = (Get-Date).AddDays(-365)
$anyReadOnly = $false
$items = Get-ChildItem $Path -File -ErrorAction SilentlyContinue
foreach($item in $items)
{
if($item.LastWriteTime -ge $date){
$false
continue
}
if($item.IsReadOnly -eq $false){
$anyReadOnly = $true
}
}
$anyReadOnly
}
This also can be simplified:
function FolderInactive
{
Param([string]$Path)
$date = (Get-Date).AddYears(-1)
$null -ne (Get-ChildItem $Path -File -ErrorAction SilentlyContinue |
Where {$_.LastWriteTime -ge $date -and $_.IsReadOnly})
}
I have written the following script using a post on this forum. The script deletes the files which are older than 15 days:
cls
$servers = Get-Content server.txt
$limit = (Get-Date).AddDays(-15)
$path = "E:\CheckDBOutput"
ForEach ($line in $servers)
{
Invoke-Command -cn $line -ScriptBlock {
Get-ChildItem -Path $path -Recurse -Force | Where-Object {
!$_.PSIsContainer -and $_.CreationTime -lt $limit
} | Remove-Item -Force
}
}
The script is executing fine, but no files are getting deleted.
As already mentioned in the comments, you need to pass the parameters to use inside the scriptblock as arguments to the -ArgumentList parameter:
$RemoveOldFiles = {
param(
[string]$path,
[datetime]$limit
)
Get-ChildItem -Path $path -Recurse -Force | Where-Object {
!$_.PSIsContainer -and $_.CreationTime -lt $limit
} | Remove-Item -Force
}
$servers = Get-Content server.txt
$limit = (Get-Date).AddDays(-15)
$path = "E:\CheckDBOutput"
ForEach ($line in $servers)
{
Invoke-Command -cn $line -ScriptBlock $RemoveOldFiles -ArgumentList $path,$limit
}
Remove-Item does not exist in PowerShell version 1.0 - make sure your destination servers have v2.0 or better installed.
I'm trying to feed my function with some variables and when no variable is given it should use a default value of 30 for OlderThanDays. For one reason or another this is not working out as I'd expected.
I fixed my problem in the ForEach loop by using if($_.B -ne $null) {$OlderThanDays=$_.B} else {$OlderThanDays="30"} But I don't think this is best practice. Can anyone tell me why [Int]$OlderThanDays=30 isn't working?
Problem: When adding a line in my csv-file without defining the OlderThanDaysvariable, the default of 30 days is not used and the files are just deleted...
Thank you for your help.
CSV file:
# Correct input formats are:
#
# ServerName, LocalPath, OlderThanDays
# Ex: server, E:\SHARE\Target, 10
# Ex: server, E:\CBR\SHARE\Target
#
# UNC-Path, OlderThanDays
# Ex: \\domain\SHARE\Target, 20
# Ex: \\domain\Target
#
# If no 'OlderThanDays' is provided, a default of 30 days will be used
# Commenting out can be done with '#'
# ______________________________________________________________________
SERVER1, E:\SHARE\Target
\\domain\SHARE\Target2
Full script:
#__________________________________________________________________________________________________________________________________
$ImportFile = "S:\Input\Scheduled Task\Auto_Clean.csv"
$Password = cat "S:\Input\pwd.txt" | ConvertTo-SecureString -Force
$UserName = "domain\me"
#__________________________________________________________________________________________________________________________________
# Scriptblock for running the function in a job
$JobCall = {
# Function that removes files older than x days in all subfolders
Function Delete-OldFiles {
[CmdletBinding(SupportsShouldProcess=$True)]
Param(
[Parameter(Mandatory=$True,Position=1)]
[ValidateScript({Test-Path $_})]
[String]$Target,
[Parameter(Mandatory=$False,Position=2)]
[Int]$OlderThanDays=30,
[Parameter(Mandatory=$False,Position=3)]
[String]$Server,
[switch]$CleanFolders
)
#__________________________________________________________________________________________________________________________________
# Create logfile
$TempDate = (get-date).ToString("dd-MM-yyyy")
$TempFolderPath = $Target -replace '\\','_'
$TempFolderPath = $TempFolderPath -replace ':',''
$TempFolderPath = $TempFolderPath -replace ' ',''
$script:LogFile = "\\DEUSTHEIDIT02\Log\Scheduled Task\Auto_Clean\$Server - $TempFolderPath - $TempDate.log"
#__________________________________________________________________________________________________________________________________
# Check the version of PowerShell
if ($PSVersionTable.PSVersion.Major -ge "3") {
# PowerShell 3+ Remove files older than (FASTER)
Get-ChildItem -Path $Target -Recurse -File |
Where-Object { $_.CreationTime -lt (Get-Date).AddDays(-$OlderThanDays) } |
ForEach {
$Item = $_.FullName
Remove-Item $Item -Recurse -Force -ErrorAction SilentlyContinue
# Log succes/failure
$Timestamp = (Get-Date).ToShortDateString()+" | "+(Get-Date).ToLongTimeString()
if (Test-Path $Item) {
"$Timestamp | FAILLED: $Server $Item (IN USE)"
}
else {
"$Timestamp | REMOVED: $Server $Item"
}
} | Tee-Object $LogFile -Append -Verbose}
Else {
# PowerShell 2 Remove files older than
Get-ChildItem -Path $Target -Recurse |
Where-Object { !$_.PSIsContainer -and $_.CreationTime -lt (Get-Date).AddDays(-$OlderThanDays) } |
ForEach {
$Item = $_.FullName
Remove-Item $Item -Recurse -Force -ErrorAction SilentlyContinue
# Log succes/failure
$Timestamp = (Get-Date).ToShortDateString()+" | "+(Get-Date).ToLongTimeString()
if (Test-Path $Item) {
Write-Host "$Timestamp | FAILLED: $Server $Item (IN USE)"
"$Timestamp | FAILLED: $Server $Item (IN USE)"
}
else {
Write-Host "$Timestamp | REMOVED: $Server $Item"
"$Timestamp | REMOVED: $Server $Item"
}
} | Out-File $LogFile -Append }
#__________________________________________________________________________________________________________________________________
# Switch -CleanFolders deletes empty folders older than x days
if ($CleanFolders) {
# Check the version of PowerShell
if ($PSVersionTable.PSVersion.Major -ge "3") {
# PowerShell 3+ Remove empty folders older than (FASTER)
Get-ChildItem -Path $Target -Recurse -Force -Directory -ErrorAction SilentlyContinue |
Where-Object { $_.CreationTime -lt (Get-Date).AddDays(-$OlderThanDays) } |
Where-Object { (Get-ChildItem -Path $_.FullName -Recurse -Force -File) -eq $null } |
ForEach {
$Item = $_.FullName
Remove-Item $Item -Recurse -Force -ErrorAction SilentlyContinue
# Log succes/failure
$Timestamp = (Get-Date).ToShortDateString()+" | "+(Get-Date).ToLongTimeString()
if (Test-Path $Item) {
"$Timestamp | FAILLED: $Server $Item (IN USE)"
}
else {
"$Timestamp | REMOVED: $Server $Item"
}
} | Tee-Object $LogFile -Append
}
else {
# PowerShell 2 Remove empty folders older than
Get-ChildItem -Path $Target -Recurse -Force -ErrorAction SilentlyContinue |
Where-Object { $_.PSIsContainer -and (Get-ChildItem -Path $_.FullName -Recurse -Force | Where-Object { !$_.PSIsContainer }) -eq $null } |
Where-Object { $_.CreationTime -lt (Get-Date).AddDays(-$OlderThanDays) } |
ForEach {
$Item = $_.FullName
Remove-Item $Item -Recurse -Force -ErrorAction SilentlyContinue
# Log succes/failure
$Timestamp = (Get-Date).ToShortDateString()+" | "+(Get-Date).ToLongTimeString()
if (Test-Path $Item) {
Write-Host "$Timestamp | FAILLED: $Server $Item (IN USE)"
"$Timestamp | FAILLED: $Server $Item (IN USE)"
}
else {
Write-Host "$Timestamp | REMOVED: $Server $Item"
"$Timestamp | REMOVED: $Server $Item"
}
} | Out-File $LogFile -Append
}
}
}
# Lact command of the ScriptBlock: Call the magic to happen
Delete-OldFiles $args[0] $args[1] $args[2] -CleanFolders:$args[3]
}
#__________________________________________________________________________________________________________________________________
# Read input file and ignore all lines starting with #
$File = (Import-Csv -Path $ImportFile -Header "A", "B", "C", "D" | Where { $_.A -NotLike "#*" } )
#__________________________________________________________________________________________________________________________________
# If the UNC Path is provided we will run the script locally else it wil be run on the remote server as a job
Foreach ($_ in $File) {
# Define input format & default values
if ($_.A -like "\\*") {
$Server="UNC"
$Target=$_.A
$OlderThanDays=$_.B
$CleanFolders=$_.C
}
else {
$Server=$_.A
$Target=$_.B
$OlderThanDays=$_.C
$CleanFolders=$_.D
}
# Call the scriptblock with the function to run locally or on the remote server
if ($Server -eq "UNC")
{
Write-Host "UNC Path detected: $Target, $OlderThanDays" -ForegroundColor Yellow
Start-Job -ScriptBlock $JobCall -ArgumentList ($Target, $OlderThanDays, $Server) -Name DelFiles
}
else
{
Write-Host "Local path detected: $Server, $Target, $OlderThanDays" -ForegroundColor Cyan
$Credentials = New-Object System.Management.Automation.PSCredential -ArgumentList $UserName,$Password
Invoke-Command -ScriptBlock $JobCall -ArgumentList ($Target, $OlderThanDays, $Server) -ComputerName "$Server.domain" -Authentication Credssp -Credential $Credentials -AsJob -JobName DelFiles
# Delete empty folders: Invoke-Command -ScriptBlock $JobCall -ArgumentList ($Target, $OlderThanDays, $Server, $true) -ComputerName "$Server.domain" -Authentication Credssp -Credential $Credentials -AsJob -JobName DelFiles
}
}
The parameter can't be both Mandatory and have default value. Former is checked first, and if it's set to $true, than default value is simply ignored. If you want to make sure that users can't specify empty value, just use [ValidateNotNullorEmpty()] validation, and make parameter optional.
So I had a similar problem. After months keeping a module with personal functions for day-to-day activities making extensive use of default values and, in many cases, mandatory parameters with initialized default values, suddenly my functions stopped working.
After writing a new one, calling it with explicit parameters would result in the parameters values being empty in the function execution context.
After restarting powershell console for some times I decided to reboot the machine, and everything went back to normal. Go figure.
I'm trying to delete all files (not folders) in %TEMP% which are older than 30 days. The problem is that some files are in use by a program so they can not be deleted. I tried to solve the problem as follow:
function IsFileLocked($filePath){
#write-host $filePath
Rename-Item $filePath $filePath -ErrorVariable errs -ErrorAction SilentlyContinue
$errs.Count
if ($errs.Count -ne 0)
{
return $true #File is locked
}
else
{
return $false #File is not locked
}
}
$Path= "$env:temp"
if ((Test-Path -Path $Path) -ieq $true)
{
$Daysback = '-30'
$CurrentDate = Get-Date
$DatetoDelete = $CurrentDate.AddDays($Daysback)
get-childitem $Path -recurse | Where-Object {$_.LastWriteTime -lt $DatetoDelete } |
Where-Object {$_.PSIsContainer -eq $False }| Where-Object {(IsFileLocked -filePath "($_)") -eq $false }# | remove-item -force #-WhatIf
}
The problem is that (IsFileLocked -filePath "($_)") -eq $false doesn't return any element.
Is it possible that get-childitem blocks the files, so that all of them are locked when I run get-childitem?
Any other ideas how to solve this problem?
How about just removing files older than 30 days and ignore the errors:
$old = (Get-Date).AddDays(-30)
Get-ChildItem $env:TEMP -Recurse |
Where-Object {!$_.PSIsContainer -and $_.LastWriteTime -lt $old } |
Remove-Item -Force -ErrorAction SilentlyContinue