PowerShell run script simultaneously - powershell

I created a PowerShell script to remove all files and folders older than X days. This works perfectly fine and the logging is also ok. Because PowerShell is a bit slow, it can take some time to delete these files and folders when big quantities are to be treated.
My questions: How can I have this script ran on multiple directories ($Target) at the same time?
Ideally, we would like to have this in a scheduled task on Win 2008 R2 server and have an input file (txt, csv) to paste some new target locations in.
Thank you for your help/advise.
The script
#================= VARIABLES ==================================================
$Target = \\share\dir1"
$OlderThanDays = "10"
$Logfile = "$Target\Auto_Clean.log"
#================= BODY =======================================================
# Set start time
$StartTime = (Get-Date).ToShortDateString()+", "+(Get-Date).ToLongTimeString()
Write-Output "`nDeleting folders that are older than $OlderThanDays days:`n" | Tee-Object $LogFile -Append
Get-ChildItem -Directory -Path $Target |
Where-Object { $_.LastWriteTime -lt (Get-Date).AddDays(-$OlderThanDays) } | ForEach {
$Folder = $_.FullName
Remove-Item $Folder -Recurse -Force -ErrorAction SilentlyContinue
$Timestamp = (Get-Date).ToShortDateString()+" | "+(Get-Date).ToLongTimeString()
# If folder can't be removed
if (Test-Path $Folder)
{ "$Timestamp | FAILLED: $Folder (IN USE)" }
else
{ "$Timestamp | REMOVED: $Folder" }
} | Tee-Object $LogFile -Append # Output folder names to console & logfile at the same time
# Set end time & calculate runtime
$EndTime = (Get-Date).ToShortDateString()+", "+(Get-Date).ToLongTimeString()
$TimeTaken = New-TimeSpan -Start $StartTime -End $EndTime
# Write footer to log
Write-Output ($Footer = #"
Start Time : $StartTime
End Time : $EndTime
Total runtime : $TimeTaken
$("-"*79)
"#)
# Create logfile
Out-File -FilePath $LogFile -Append -InputObject $Footer
# Clean up variables at end of script
$Target=$StartTime=$EndTime=$OlderThanDays = $null

One way to achieve this would be to write an "outer" script that passes the directory-to-be-cleaned, into the "inner" script, as a parameter.
For your "outer" script, have something like this:
$DirectoryList = Get-Content -Path $PSScriptRoot\DirList;
foreach ($Directory in $DirectoryList) {
Start-Process -FilePath powershell.exe -ArgumentList ('"{0}\InnerScript.ps1" -Path "{1}"' -f $PSScriptRoot, $Directory);
}
Note: Using Start-Process kicks off a new process that is, by default, asynchronous. If you use the -Wait parameter, then the process will run synchronously. Since you want things to run more quickly and asynchronously, omitting the -Wait parameter should achieve the desired results.
Invoke-Command
Alternatively, you could use Invoke-Command to kick off a PowerShell script, using the parameters: -File, -ArgumentList, -ThrottleLimit, and -AsJob. The Invoke-Command command relies on PowerShell Remoting, so that must enabled, at least on the local machine.
Add a parameter block to the top of your "inner" script (the one you posted above), like so:
param (
[Parameter(Mandatory = $true)]
[string] $Path
)
That way, your "outer" script can pass in the directory path, using the -Path parameter for the "inner" script.

Related

PowerShell Core script runs but is not executed by Task Scheduler?

I am trying to schedule a PowerShell Core 7.2 script to run on Windows Server 2012R2.
The script runs manually, without any errors, from the server and the Task Scheduler runs the task. In the History, I can see Task Completed
The issue is that the script is not executed. It is supposed to move the files and files are not moving which means the script was not executed.
Settings of the Task Scheduler that are selected are as follows:
General - Run whether user is logged on or not, Run with the highest privileges.
Actions -> Action Start a Program
Actions -> Program/Script "C:\Program Files\PowerShell\7\pwsh.exe" (location of pwsh.exe)
Actions -> Add Arguments -Executionpolicy Bypass -File "R:\Public\IT\Vantage_Utilities\CNC_Scripts\File Transfer\Fastems\CNC_File_Transfer_Fastems.ps1"
Location -> Name of the local machine
I am not really sure what is going wrong here.
EDIT
I am thinking there is an issue with the script. Because there is another script set up to be executed with PS Core and Task Scheduler. I am going to post the script here. It is a simple batch file that moves all the contents of one folder from one server to another. I achieve this in two functions. Function MoveFiles moves all the contents of the parent folder(excluding the subfolder called "Mazak"). The second function,Function MoveMazakFiles function moves the contents of "Mazak" only. ( I am completely aware I could have done this using fewer lines of code but that is not the point here)
Code:
$logPath = "\\MMS25163S1\Public\IT\Vantage_Utilities\CNC_Scripts\File Transfer\Fastems\Log.txt"
$trancriptPath = "\\MMS25163S1\Public\IT\Vantage_Utilities\CNC_Scripts\File Transfer\Fastems\LogTranscript.txt"
$getDate = Get-Date -Format "dddd MM/dd/yyyy HH:mm "
$counter = 0
$mazakCounter = 0
Start-Transcript -Path $trancriptPath -Append
Add-Content -Path $logPath -Value ("LOG CREATED $getDate") -PassThru
#Sources
$srcMca = "\\MMS25163S1\Public\NcLib\FromNC\*"
$srcMcaNameChg ="\\MMS25163S1\Public\NcLib\FromNC"
$srcMazak= "\\MMS25163S1\Public\NcLib\FromNC\Mazak\*"
$srcMcaNameChgMazak = "\\MMS25163S1\Public\NcLib\FromNC\Mazak"
#Destination
$destMca = "\\Sidney2\MfgLib\RevisedPrograms\MC-A"
#Time with milliseconds
$time = (Get-Date -Format hh-mm-fff-tt).ToString()
Function MoveFiles{
Param(
[string]$src,
[string]$dest,
[string]$srcNameChange
)
Get-Item -Path $src -Exclude *Mazak* -ErrorAction SilentlyContinue | ForEach-Object{
$counter++
$fileName = $_.BaseName
$fileNameExt = $_.Name
Write-host $fileName -ForegroundColor Green
Rename-Item -Path "$srcMcaNameChg\$fileNameExt" -NewName ($fileName+"_"+"(Time-$time)"+$_.Extension);
Add-Content -Path $logPath -Value ("Name changed: Time stamp added to $fileName ") -PassThru
}
Move-Item -Path $src -Exclude *Mazak* -Destination $dest -Force
Add-Content -Path $logPath -Value ("$counter file(s) moved to $dest") -PassThru
}
MoveFiles -src $srcMca -dest $destMca -srcNameChange $srcMcaNameChg
Function MoveMazakFiles{
Param(
[string]$srcMazak,
[string]$dest,
[string]$srcNameChange
)
Get-ChildItem $srcMazak -Recurse -ErrorAction SilentlyContinue | ForEach-Object{
$mazakCounter++
$fileName = $_.BaseName
$fileNameExt = $_.Name
Write-host $fileName -ForegroundColor Green
Rename-Item -Path "$srcMcaNameChgMazak\$fileNameExt" -NewName ($fileName+"_"+"(Time-$time)"+$_.Extension);
}
Move-Item -Path $srcMazak -Destination $dest -Force
Add-Content -Path $logPath -Value ("$mazakCounter file(s) from Mazak folder moved to $dest") -PassThru
}
MoveMazakFiles -srcMazak $srcMazak -dest $destMca -srcNameChange $srcMcaNameChg
Stop-Transcript
When setting the scheduled task, under Action -> Start a Program -> Program/Script. Call powershell and pass the script as the parameter
Like
powershell -File "R:\Public\IT\Vantage_Utilities\CNC_Scripts\File Transfer\Fastems\CNC_File_Transfer_Fastems.ps1"

Powershell runs script from previous edit

I am practicing Powershell in VSCode.
I have noticed that when I hit F5 to run my script, Powershell runs the previous version of the script (before my changes). I am definitely saving the file after my edits.
So, for example, I might change the text in this line:
Add-Content -Path "D:\Text_File.txt" -value "$_ - MP3 File count is: $Cnt"
But it will continue to show the same output until I run it a second time.
This is the script I am using in case it is relevant:
Param(
[string]$Path = 'D:\WMA - BU of wma music files',
[string]$DestinationPath = "D:\WMA-Only",
[string]$OutputFileDest = "D:\Text_File.txt"
)
$MainDirList = Get-ChildItem -Path $Path -Directory | ForEach-Object {$_.FullName}
$MainDirList | Out-File -FilePath D:\Text_File.txt
$MainDirList | WriteFolder
function WriteFolder
{
process
{
$FilteredList = Get-ChildItem -Path $_ -Force -Recurse -Filter "*.mp3"
$Cnt = $FilteredList.count
Add-Content -Path "D:\Text_File.txt" -value "$_ - MP3 File count is: $Cnt"
}
}
Visual Studio code version is 1.62.2
Powershell extension is v2021.10.2
I am wondering if I am meant to clear variables or the like at the end of a script?
I am viewing the output in the latest version of notepad++
Could it be because you've defined one function inside another function? Normally when you run a script, it finds any Functions and pre-processes them before running the script. But in your case, it only processes the outer function, which only creates the inner function when it's run.
I'm not sure what your parent function is called, but you could try just having the two functions side-by-side:
Function myFunction{
Param(
[string]$Path = 'D:\WMA - BU of wma music files',
[string]$DestinationPath = "D:\WMA-Only",
[string]$OutputFileDest = "D:\Text_File.txt"
)
Process{
$MainDirList = Get-ChildItem -Path $Path -Directory | ForEach-Object {$_.FullName}
$MainDirList | Out-File -FilePath D:\Text_File.txt
$MainDirList | WriteFolder
}
}
function WriteFolder
{
process
{
$FilteredList = Get-ChildItem -Path $_ -Force -Recurse -Filter "*.mp3"
$Cnt = $FilteredList.count
Add-Content -Path "D:\Text_File.txt" -value "$_ - MP3 File count is: $Cnt"
}
}
Or else you could move the WriteFolder function into its own Powershell module (*.psm1) and load it in your main script:
Import-Module "WriteFolder.psm1" -Force
The -Force parameter is optional, but if you're regularly changing the module file you'll need it to ensure the updates are loaded, rather than just caching the old version of the module.

creating logfile for a powershell script day wise

$dateTime = Get-Date -Format "yyyyMMdd"
$Logfile = $logfile + $dateTime +".log"
if((Get-ChildItem $Logfile).CreationTime.Date -ne (Get-Date).Date)
{
Write-Host "creating new"
New-Item -Path $Logfile -ItemType File -Force
}
else
{
Write-Host "existing"
}
## This function facilitates in capturing various events into a log file when the script will run
function WriteLog
{
param([string]$Message)
filter timestamp {"$(Get-Date -Format G) $_"}
$Message = $Message | timestamp
Add-content $Logfile -value $Message
}
I am using this small code which will create log file per day if doesn't exist. It append log messages whenever Writelog function is triggered.
Problem facing --> this is working as per expectations for maximum 4 consecutive runs and After this the script is running fine, but not appending any message to the logfile.
If it was me I would define your log path variable once and then use throughout your script. You seem to be trying to access the same name by different methods which will be prone to errors. Your Write-Log method for example should take two parameters: Message and LogFile.
For an easier way to create logs using a Transcript, I have a logs sub folder and I just top and tail my script with the following:
# Start logging
#######################################################################
$log_path = $PSScriptRoot | Join-Path -ChildPath 'Logs'
$script_name = $PSCommandPath -replace '^.*\\([^\\]+)\.ps1$', '$1'
$log_name = '{0}_{1:yyyyMMddhhmmss}.log' -f $script_name, (Get-Date)
$log = $log_path | Join-Path -ChildPath $log_name
Start-Transcript -Path $log -Append -IncludeInvocationHeader
# Do stuff
# Use Write-Host for log comments
# Captures errors without additional code
# Stop logging
#######################################################################
Stop-Transcript

Powershell Script to Move Files to new server by last access date

I'm new to Powershell. I have 80 servers that I need to connect to and run a Pshell script on remotely to find files recursively in one share by last access date and move them to another \server\share for archiving purposes. I also need the file creation, last accessed etc. timestamps to be preserved.
I would welcome any help please
thank you
You need to test this thoroughly before actually using it on all 80 servers!
What you could do if you want to use PowerShell on this is to use Invoke-Command on the servers adding admin credentials so the script can both access the files to move as well as the destination Archive folder.
I would suggest using ROBOCOPY to do the heavy lifting:
$servers = 'Server1', 'Server2', 'Server3' # etcetera
$cred = Get-Credential -Message "Please supply admin credentials for archiving"
$scriptBlock = {
$SourcePath = 'D:\StuffToArchive' # this is the LOCAL path on the server
$TargetPath = '\\NewServer\ArchiveShare' # this is the REMOTE path to where the files should be moved
$LogFile = 'D:\ArchivedFiles.txt' # write a textfile with all fie fullnames that are archived
$DaysAgo = 130
# from a cmd box, type 'robocopy /?' to see all possible switches you might want to use
# /MINAGE:days specifies the LastWriteTime
# /MINLAD:days specifies the LastAccessDate
robocopy $SourcePath $TargetPath /MOVE /MINLAD:$DaysAgo /COPYALL /E /FP /NP /XJ /XA:H /R:5 /W:5 /LOG+:$logFile
}
Invoke-Command -ComputerName $servers -ScriptBlock $scriptBlock -Credential $cred
If you want to do all using just PowerShell, try something like this:
$servers = 'Server1', 'Server2', 'Server3' # etcetera
$cred = Get-Credential -Message "Please supply admin credentials for archiving"
$scriptBlock = {
$SourcePath = 'D:\StuffToArchive' # this is the LOCAL path on the server
$TargetPath = '\\NewServer\ArchiveShare' # this is the REMOTE path to where the files should be moved
$LogFile = 'D:\ArchivedFiles.txt' # write a textfile with all fie fullnames that are archived
$refDate = (Get-Date).AddDays(-130).Date # the reference date set to midnight
# set the ErrorActionPreference to Stop, so exceptions are caught in the catch block
$OldErrorAction = $ErrorActionPreference
$ErrorActionPreference = 'Stop'
# loop through the servers LOCAL path to find old files and move them to the remote archive
Get-ChildItem -Path $SourcePath -File -Recurse |
Where-Object { $_.LastAccessTime -le $refDate } |
ForEach-Object {
try {
$target = Join-Path -Path $TargetPath -ChildPath $_.DirectoryName.Substring($SourcePath.Length)
# create the folder in the archive if not already exists
$null = New-Item -Path $target -ItemType Directory -Force
$_ | Move-Item -Destination $target -Force
Add-Content -Path $LogFile -Value "File '$($_.FullName)' moved to '$target'"
}
catch {
Add-Content -Path $LogFile -Value $_.Exception.Message
}
}
$ErrorActionPreference = $OldErrorAction
}
Invoke-Command -ComputerName $servers -ScriptBlock $scriptBlock -Credential $cred

Powershell - Monitoring live log file Q2

I had made an initial question here Which was answered but as i move along in my task I'm running into another problem.
Summary: I have a log file that's being written to via a serial device. I'm wanting to monitor this log file for particular strings (events) and when they happen i want to write those strings to a separate file.
Executing this one off does what I'm looking for:
$p = #("AC/BATT_PWR","COMM-FAULT")
$fileName = "SRAS_$(Get-Date -format yyyy-MM-dd).log"
$fullPath = "C:\temp\SRAS\$fileName"
Get-Content $fullpath -tail 1 -Wait | Select-String -Pattern $p -SimpleMatch | Out-File -Filepath C:\temp\SRAS\sras_pages.log -Append
The problem is the logfile gets a datestamp, putty saves it as SRAS_yyyy-mm-dd.log. So when the clock passes midnight this will no longer be looking at the correct file.
I found this post on SO which is exactly what I want to do, the OP claims it works for him. I modified it slightly for my purposes but it doesn't write events matching the desired strings to sras_pages.log
This is the 'modified' code:
while($true)
{
$now = Get-Date
$fileName = "SRAS_$(Get-Date -format yyyy-MM-dd).log"
$fullPath = "C:\temp\SRAS\$fileName"
$p = #("AC/BATT_PWR","COMM-FAULT")
Write-Host "[$(Get-Date)] Starting job for file $fullPath"
$latest = Start-Job -Arg $fullPath -ScriptBlock {
param($file)
# wait until the file exists, just in case
while(-not (Test-Path $fullpath)){ sleep -sec 10 }
Get-Content $file -Tail 1 -wait | Select-String -Pattern $p |
foreach { Out-File -Filepath "C:\temp\SRAS\sras_pages.log" -Append }
}
# wait until day changes, or whatever would cause new log file to be created
while($now.Date -eq (Get-Date).Date){ sleep -Sec 10 }
# kill the job and start over
Write-Host "[$(Get-Date)] Stopping job for file $fullPath"
$latest | Stop-Job
}
If I execute just the Get-Content segment of that code it does exactly what I'm looking for. I can't figure out what the issue is.
TIA for advice.
Here is a few suggested changes that should make it work:
$p does not exist within the job, add it as a parameter ($pattern in my example)
You are referring to $fullpath within your job (row 13), it should be $file.
Add parameter -SimpleMatch to select-string to search for literal strings instead of regular expressions. (This is not needed but will come in handy if you change search pattern)
Referring to $pattern instead of $p (see 1)
Skip the foreach on row 16.
Like this:
while($true)
{
$now = Get-Date
$fileName = "SRAS_$(Get-Date -format yyyy-MM-dd).log"
$fullPath = "C:\temp\SRAS\$fileName"
$p = #("AC/BATT_PWR","COMM-FAULT")
Write-Host "[$(Get-Date)] Starting job for file $fullPath"
$latest = Start-Job -Arg $fullPath, $p -ScriptBlock {
param($file,$pattern)
# wait until the file exists, just in case
while(-not (Test-Path $file)){ sleep -sec 10 }
Get-Content $file -Tail 1 -wait | Select-String -Pattern $pattern -SimpleMatch |
Out-File -Filepath "C:\temp\SRAS\sras_pages.log" -Append
}
# wait until day changes, or whatever would cause new log file to be created
while($now.Date -eq (Get-Date).Date){ sleep -Sec 10 }
# kill the job and start over
Write-Host "[$(Get-Date)] Stopping job for file $fullPath"
$latest | Stop-Job
}