I have few folders in which my script runs and triggers job for each folder. Imagine if I have 10 subfolders inside the main folder and I want to limit the number of jobs running to at a time to be 5. how can I do that? I will put my code below.
function ThrottleJob([int]$MaxJob) {
do {
$jobs = Get-Job -state "Running"
Start-Sleep -Seconds 1
} while ($jobs.Count -gt $MaxJob)
}
$folders = #()
$folders += (Get-Item $path).FullName
$folders += Get-ChildItem -Path $path -Recurse -Directory
foreach ($folder in $folders) {
if (Get-ChildItem -Path $folder) {
ThrottleJob -MaxJob 5
Start-Job -InitializationScript $allJobs -ScriptBlock { runJob -path $args[0]} -ArgumentList $folder | Out-Null
}
}
Could anyone please help me out?
Here you can an example for setting a threshold for Start-Job, I do believe a normal foreach loop will be faster than this.
using namespace System.Collections.Generic
using namespace System.Management.Automation
$maxJobs = 5
$path = "$HOME\Documents"
$folders = #(
(Get-Item $path).FullName
(Get-ChildItem -Path $path -Recurse -Directory).FullName
)
$jobs = [List[Job]]::new()
foreach($folder in $folders)
{
$job = Start-Job {
Get-ChildItem $using:folder -File | Select-Object DirectoryName, Name
}
$jobs.Add($job)
while(($jobs.State -eq 'Running').Count -ge $maxJobs)
{
Clear-Host
Get-Job -State Running | Format-Table -AutoSize
Start-Sleep -Seconds 1
}
}
$result = Receive-Job $jobs -Wait -AutoRemoveJob
$result | Format-Table
As you can see, the code is awfully complicated. This is how it would look using Start-ThreadJob, which will most likely be faster than a foreach loop and will consume less memory than Start-Job.
$maxJobs = 5
$path = "$HOME\Documents"
$folders = #(
(Get-Item $path).FullName
(Get-ChildItem -Path $path -Recurse -Directory).FullName
)
$jobs = foreach($folder in $folders)
{
Start-ThreadJob {
Get-ChildItem $using:folder -File | Select-Object DirectoryName, Name
} -ThrottleLimit $maxJobs
}
$result = Receive-Job $jobs -Wait -AutoRemoveJob
$result | Format-Table
$Servers = Get-content 'D:\utils\Backup\SBX servers.txt'
foreach($Path in $Servers){
$Path = "E:\Backup"
$result = Get-ChildItem E:\Backup\*_DB.zip -Recurse -Force -File |Format-Wide| Where-Object { $_.LastWriteTime -gt (get-date).AddDays(-1)}
$result
}
you were not far from the solution, but there are multiple problem with your syntax. First you should not "filter" (where-object), after "formating" (Format-wide).
foreach($Path in $Servers){
$Path = "E:\Backup"
$result = Get-ChildItem E:\Backup*DB.zip -Recurse -Force -File | Where-Object { $_.LastWriteTime -gt (get-date).AddDays(-1)}
$result | Format-Wide
}
you were missing the "underscore" in your filter too: "$." -> "$_."
$computers = Get-Content "U:\listofremotecomputers.txt"
$files = Get-Content "U:\lookforthisfiles.txt"
foreach ($computer in $computers) {
foreach ($file in $files) {
Get-ChildItem -Recurse -Force \\$computer\ -ErrorAction SilentlyContinue |`
Where-Object {
($_.PSIsContainer -eq $false) -and ( $_.Name -eq "$files")
} |`
Select-Object Name,Directory |`
Export-Csv U:\FoundFiles.csv -nti -append
}
}
How can I list the files, and able to locate them in a remote server?
I have tried the code below, not working still. I just want to locate what ever file is define in lookforthisfiles.txt, instead of looking at them one by one.
ForEach ($computers in $computers) { invoke-command -computername $server {Get-ChildItem -Recurse -Force $files }}
I'm trying to create a powershell script that searches a users C drive for a certain file extension, and then writes a file to a network share if it finds one. The script is launched from the last line of a logon script that reads like this:
powershell.exe -ExecutionPolicy Bypass -windowstyle hidden -file "\\servername\Path\To\File.ps1"
And my powershell script looks like this:
$hostname = HostName
Get-ChildItem -Path C:\*.* -Filter $file -Recurse -Force -ErrorAction
SilentlyContinue | Out-File \\servername\Path\To\Results\$hostname\file.txt
If ((Get-Content "\\servername\Path\To\Results\$hostname\file.txt") -eq $Null) {
Remove-Item \\servername\Path\To\Results\$hostname\file.txt
}
Exit
The script runs perfectly fine on my machine even when I load it from the network share but whenever another computer runs it, it never produces an Out File. And I don't think it is even searching.
What am I doing wrong? I thought it was execution policy, but I've set the logon script to bypass it. I don't understand why it isn't working.
[edit]
I've now got the script working sometimes on Windows 10 machines. But it doesn't work at all on Windows 7. I've even tried running
Get-ChildItem C:\*.pst -Recurse
directly from a powershell command prompt, and it just fails silently and doesn't search for anything. Isn't Get-ChildItem a powershell 1 command?
Hello. If you do like this: Get-ChildItem -Path C:*.* -Filter $file
-Recurse -Force the text file output will be enough to weigh.
You can try to check the access to the network folder for the current
user:if access explicitly set, and write access exists, then you can
record a file with the content. Otherwise, it can create folder test
on the local machine which will create the file, indicating that there
is no access. How is this way:
Set-ExecutionPolicy remotesigned -Scope CurrentUser -Force| Out-Null
$user = [System.Environment]::UserName
$hostname = [System.Environment]::MachineName
try {
$accs = Get-ACL -Path "\\server\sharedfolder\Results\"
foreach ($access in $accs) {
$obj = New-Object -TypeName PSObject -Property #{
Access = $accs.Access
}
$obj1 = $obj | select -ExpandProperty Access
for ($i = 0 ; $i -le $obj1.Count ; $i ++ )
{
if (!($obj1[$i].IdentityReference -like "*Users*" -or $obj1[$i].IdentityReference -like "*$user*")) {
if (!(Test-Path "c:\test")) {
md c:\test
$s = "user access from group"
$s | out-file C:\test\ErrInfo.csv
}
else {
$s = "user access from group"
$s | out-file C:\test\ErrInfo.csv
}
}
if ($obj1[$i].IdentityReference -like "*Users*" -or $obj1[$i].IdentityReference -like "*$user*") {
if ($obj1[$i].FileSystemRights -like "*ReadAndExecute*")
{
if (!(Test-Path "c:\test")) {
md c:\test
$s = "Premission only ReadAndExecute"
$s | out-file C:\test\ErrInfo_rex.csv
}
else {
$s = "Premission only ReadAndExecute"
$s | out-file C:\test\ErrInfo_rex.csv
}
}
if ($obj1[$i].FileSystemRights -like "*FullControl*" -and $obj1[$i].AccessControlType -like "*Allow*" -or $obj1[$i].FileSystemRights -like "*Modify*" -and $obj1[$i].AccessControlType -like "*Allow*")
{
if (!(Test-Path "\\server\sharedfolder\Results\$hostname"))
{
md "\\server\sharedfolder\Results\$hostname"
Get-ChildItem -Path C:\testpatch\*.* -Filter $file -Recurse -Force -ErrorAction SilentlyContinue | Out-File "\\server\sharedfolder\Results\$hostname\file.txt"
}
else {
Get-ChildItem -Path C:\testpatch\*.* -Filter $file -Recurse -Force -ErrorAction SilentlyContinue | Out-File "\\server\sharedfolder\Results\$hostname\file.txt"
}
}
}
}
}
}
catch {
if (!(Test-Path "c:\test")) {
md c:\test
$s = "--NoAccess--"
$s | out-file C:\test\ErrInfo_noaccess.csv
}
else {
$s = "--NoAccess--"
$s | out-file C:\test\ErrInfo_noaccess.csv
}
}
Or you can do something like this (whiteout EXIT):
Set-ExecutionPolicy remotesigned -Scope CurrentUser -Force| Out-Null
$hostname = [System.Environment]::MachineName
if (!(Test-Path "\\server\sharedfolder\Results\$hostname")) {
md "\\server\sharedfolder\Results\$hostname" | Out-Null
If ((Get-ChildItem -Path C:\test\*.* -Filter $file -Recurse -Force -ErrorAction SilentlyContinue).Count -ne "0") {
Get-ChildItem -Path C:\test\*.* -Filter $file -Recurse -Force -ErrorAction SilentlyContinue | Out-File "\\server\sharedfolder\Results\$hostname\file.txt"
}
}
elseif (Test-Path "\\server\sharedfolder\Results\$hostname") {
If ((Get-ChildItem -Path C:\test\*.* -Filter $file -Recurse -Force -ErrorAction SilentlyContinue).Count -ne "0") {
Get-ChildItem -Path C:\test\*.* -Filter $file -Recurse -Force -ErrorAction SilentlyContinue | Out-File "\\server\sharedfolder\Results\$hostname\file.txt"
}
}
I'm trying to feed my function with some variables and when no variable is given it should use a default value of 30 for OlderThanDays. For one reason or another this is not working out as I'd expected.
I fixed my problem in the ForEach loop by using if($_.B -ne $null) {$OlderThanDays=$_.B} else {$OlderThanDays="30"} But I don't think this is best practice. Can anyone tell me why [Int]$OlderThanDays=30 isn't working?
Problem: When adding a line in my csv-file without defining the OlderThanDaysvariable, the default of 30 days is not used and the files are just deleted...
Thank you for your help.
CSV file:
# Correct input formats are:
#
# ServerName, LocalPath, OlderThanDays
# Ex: server, E:\SHARE\Target, 10
# Ex: server, E:\CBR\SHARE\Target
#
# UNC-Path, OlderThanDays
# Ex: \\domain\SHARE\Target, 20
# Ex: \\domain\Target
#
# If no 'OlderThanDays' is provided, a default of 30 days will be used
# Commenting out can be done with '#'
# ______________________________________________________________________
SERVER1, E:\SHARE\Target
\\domain\SHARE\Target2
Full script:
#__________________________________________________________________________________________________________________________________
$ImportFile = "S:\Input\Scheduled Task\Auto_Clean.csv"
$Password = cat "S:\Input\pwd.txt" | ConvertTo-SecureString -Force
$UserName = "domain\me"
#__________________________________________________________________________________________________________________________________
# Scriptblock for running the function in a job
$JobCall = {
# Function that removes files older than x days in all subfolders
Function Delete-OldFiles {
[CmdletBinding(SupportsShouldProcess=$True)]
Param(
[Parameter(Mandatory=$True,Position=1)]
[ValidateScript({Test-Path $_})]
[String]$Target,
[Parameter(Mandatory=$False,Position=2)]
[Int]$OlderThanDays=30,
[Parameter(Mandatory=$False,Position=3)]
[String]$Server,
[switch]$CleanFolders
)
#__________________________________________________________________________________________________________________________________
# Create logfile
$TempDate = (get-date).ToString("dd-MM-yyyy")
$TempFolderPath = $Target -replace '\\','_'
$TempFolderPath = $TempFolderPath -replace ':',''
$TempFolderPath = $TempFolderPath -replace ' ',''
$script:LogFile = "\\DEUSTHEIDIT02\Log\Scheduled Task\Auto_Clean\$Server - $TempFolderPath - $TempDate.log"
#__________________________________________________________________________________________________________________________________
# Check the version of PowerShell
if ($PSVersionTable.PSVersion.Major -ge "3") {
# PowerShell 3+ Remove files older than (FASTER)
Get-ChildItem -Path $Target -Recurse -File |
Where-Object { $_.CreationTime -lt (Get-Date).AddDays(-$OlderThanDays) } |
ForEach {
$Item = $_.FullName
Remove-Item $Item -Recurse -Force -ErrorAction SilentlyContinue
# Log succes/failure
$Timestamp = (Get-Date).ToShortDateString()+" | "+(Get-Date).ToLongTimeString()
if (Test-Path $Item) {
"$Timestamp | FAILLED: $Server $Item (IN USE)"
}
else {
"$Timestamp | REMOVED: $Server $Item"
}
} | Tee-Object $LogFile -Append -Verbose}
Else {
# PowerShell 2 Remove files older than
Get-ChildItem -Path $Target -Recurse |
Where-Object { !$_.PSIsContainer -and $_.CreationTime -lt (Get-Date).AddDays(-$OlderThanDays) } |
ForEach {
$Item = $_.FullName
Remove-Item $Item -Recurse -Force -ErrorAction SilentlyContinue
# Log succes/failure
$Timestamp = (Get-Date).ToShortDateString()+" | "+(Get-Date).ToLongTimeString()
if (Test-Path $Item) {
Write-Host "$Timestamp | FAILLED: $Server $Item (IN USE)"
"$Timestamp | FAILLED: $Server $Item (IN USE)"
}
else {
Write-Host "$Timestamp | REMOVED: $Server $Item"
"$Timestamp | REMOVED: $Server $Item"
}
} | Out-File $LogFile -Append }
#__________________________________________________________________________________________________________________________________
# Switch -CleanFolders deletes empty folders older than x days
if ($CleanFolders) {
# Check the version of PowerShell
if ($PSVersionTable.PSVersion.Major -ge "3") {
# PowerShell 3+ Remove empty folders older than (FASTER)
Get-ChildItem -Path $Target -Recurse -Force -Directory -ErrorAction SilentlyContinue |
Where-Object { $_.CreationTime -lt (Get-Date).AddDays(-$OlderThanDays) } |
Where-Object { (Get-ChildItem -Path $_.FullName -Recurse -Force -File) -eq $null } |
ForEach {
$Item = $_.FullName
Remove-Item $Item -Recurse -Force -ErrorAction SilentlyContinue
# Log succes/failure
$Timestamp = (Get-Date).ToShortDateString()+" | "+(Get-Date).ToLongTimeString()
if (Test-Path $Item) {
"$Timestamp | FAILLED: $Server $Item (IN USE)"
}
else {
"$Timestamp | REMOVED: $Server $Item"
}
} | Tee-Object $LogFile -Append
}
else {
# PowerShell 2 Remove empty folders older than
Get-ChildItem -Path $Target -Recurse -Force -ErrorAction SilentlyContinue |
Where-Object { $_.PSIsContainer -and (Get-ChildItem -Path $_.FullName -Recurse -Force | Where-Object { !$_.PSIsContainer }) -eq $null } |
Where-Object { $_.CreationTime -lt (Get-Date).AddDays(-$OlderThanDays) } |
ForEach {
$Item = $_.FullName
Remove-Item $Item -Recurse -Force -ErrorAction SilentlyContinue
# Log succes/failure
$Timestamp = (Get-Date).ToShortDateString()+" | "+(Get-Date).ToLongTimeString()
if (Test-Path $Item) {
Write-Host "$Timestamp | FAILLED: $Server $Item (IN USE)"
"$Timestamp | FAILLED: $Server $Item (IN USE)"
}
else {
Write-Host "$Timestamp | REMOVED: $Server $Item"
"$Timestamp | REMOVED: $Server $Item"
}
} | Out-File $LogFile -Append
}
}
}
# Lact command of the ScriptBlock: Call the magic to happen
Delete-OldFiles $args[0] $args[1] $args[2] -CleanFolders:$args[3]
}
#__________________________________________________________________________________________________________________________________
# Read input file and ignore all lines starting with #
$File = (Import-Csv -Path $ImportFile -Header "A", "B", "C", "D" | Where { $_.A -NotLike "#*" } )
#__________________________________________________________________________________________________________________________________
# If the UNC Path is provided we will run the script locally else it wil be run on the remote server as a job
Foreach ($_ in $File) {
# Define input format & default values
if ($_.A -like "\\*") {
$Server="UNC"
$Target=$_.A
$OlderThanDays=$_.B
$CleanFolders=$_.C
}
else {
$Server=$_.A
$Target=$_.B
$OlderThanDays=$_.C
$CleanFolders=$_.D
}
# Call the scriptblock with the function to run locally or on the remote server
if ($Server -eq "UNC")
{
Write-Host "UNC Path detected: $Target, $OlderThanDays" -ForegroundColor Yellow
Start-Job -ScriptBlock $JobCall -ArgumentList ($Target, $OlderThanDays, $Server) -Name DelFiles
}
else
{
Write-Host "Local path detected: $Server, $Target, $OlderThanDays" -ForegroundColor Cyan
$Credentials = New-Object System.Management.Automation.PSCredential -ArgumentList $UserName,$Password
Invoke-Command -ScriptBlock $JobCall -ArgumentList ($Target, $OlderThanDays, $Server) -ComputerName "$Server.domain" -Authentication Credssp -Credential $Credentials -AsJob -JobName DelFiles
# Delete empty folders: Invoke-Command -ScriptBlock $JobCall -ArgumentList ($Target, $OlderThanDays, $Server, $true) -ComputerName "$Server.domain" -Authentication Credssp -Credential $Credentials -AsJob -JobName DelFiles
}
}
The parameter can't be both Mandatory and have default value. Former is checked first, and if it's set to $true, than default value is simply ignored. If you want to make sure that users can't specify empty value, just use [ValidateNotNullorEmpty()] validation, and make parameter optional.
So I had a similar problem. After months keeping a module with personal functions for day-to-day activities making extensive use of default values and, in many cases, mandatory parameters with initialized default values, suddenly my functions stopped working.
After writing a new one, calling it with explicit parameters would result in the parameters values being empty in the function execution context.
After restarting powershell console for some times I decided to reboot the machine, and everything went back to normal. Go figure.