PowerShell script speed standalone vs within Visual Code - powershell

I have a simple PowerShell script that writes to a file a list of recently added files.
$fpath = "\\DS1821\video\movies"
$file1 = "C:\Users\brian.w.williams\Desktop\RecentMovie.txt"
if (Test-Path $file1) {Remove-Item -Path $file1}
Get-ChildItem -Path "$fpath" -File -Recurse -Include "*.eng.srt" |
ForEach-Object {
$movie = $_.BaseName -replace ".eng",""
if ( ($_.LastWriteTime.Month -ge 7) -and ($_.LastWriteTime.Year -ge 2021) ) {
Write-Host $movie " = " $_.LastWriteTime
Write-Output $movie | Out-file $file1 -append;
}
}
The script works fine. But I noticed that the script runs much faster (a couple of minutes) when run within Visual Code (i.e. "Run without debugging"). When I run the script standalone (i.e. "Run with PowerShell") the script can take hours to complete. Why the difference? Is there anything I can do to speed it up? I have tried mapping the network folder, but that made no difference.

There is improvement to be made to help the code speed up.
First of all, using Write-Host and Write-Output to append to a file inside the ForEach loop is very time consuming.
Then you're using parameter -Include on Get-ChildItem, where you really want to use -Filter. A Filter is MUCH faster than Include, because the latter will only filter the filenames afterwards. -Filter however can only work on one single filename pattern, but this is exactly what you are doing here.
The if(..) can also be improved to have it compare only one value (a datetime) instead of two separate properties from LastWriteTime.
Try:
$sourcePath = '\\DS1821\video\movies'
# set a variable to an output file on your desktop
$outputFile = Join-Path -Path ([Environment]::GetFolderPath("Desktop")) -ChildPath 'RecentMovies.txt'
# set the reference date in a variable for faster comparison later
$refdate = (Get-Date -Year 2021 -Month 7 -Day 1).Date
# get the files, filter on the name ending in '*.eng.srt'
# filter more with Where-Object so you'll get only files newer than or equal to the reference date
# output objects with one calculated property ('MovieName') and the LastWriteTime
$movies = Get-ChildItem -Path $sourcePath -File -Recurse -Filter '*.eng.srt' |
Where-Object { $_.LastWriteTime -ge $refdate } |
Select-Object #{Name = 'MovieName'; Expression = {$_.BaseName -replace '\.eng\.srt$', '.srt'}},
LastWriteTime
# now output what you have collected to file.
# Set-Content creates a new file or overwrites an existing file
$movies.MovieName | Set-Content -Path $outputFile
# and to screen
$movies | ForEach-Object { Write-Host ('{0} = {1}' -f $_.MovieName, $_.LastWriteTime.ToString()) }
# or to Out-GridView if you prefer
$movies | Out-GridView -Title 'My Movies'
1. I've changed some variable names to make the code better readable
2. Since -replace uses regex, you have to escape the dots with a backslash. The regex also anchors the string to the end of the BaseName with the dollar sign ($) at the end

Related

Creating new files from multiple files that have been scanned and manipulated in powershell

First of all, I'm a complete newbie at Powershell.
I've basically compiled a script from a number google search results and it works to a certain degree, so be gentle :)
I have a number of large plain text files that need scanning, junk data needs removing, and characters need renaming. Then create a new file in the same directory
Here is the script I have for individual files, I have replaced actual keywords for something unrelated, but for testing purposes you should see what I am trying to achieve:
Get-Content C:\Temp\Tomatoes-2022-09-27.txt |
Where-Object { - $_.Contains('red') } | # Keeping only lines containing "red"
Foreach {$_ -replace "[/()]",":"}| # replacing specific characters to a colon
Where-Object { -not $_.Contains('too red') } | # removing lines containing "too red"
Set-Content C:\Temp\Tomatoes-2022-09-27Ripe.txt # saving as a new file *Ripe.txt
This works for individual files just fine but what I need to do is the same process for any file within the Temp directory.
They all have similar names other than the date.
Here's what I have compiled for all files, but it overwrites existing files rather than creating a new one and I don't know how to get it to write to new files ie Tomotoes*Ripe.txt: *being the unique date
Get-ChildItem C:\Temp\*.* -Recurse | ForEach-Object {
(Get-Content $_) |
Where-Object { - $_.Contains('red') } |
ForEach-Object { $_ -replace "[/()]", ":" } |
Where-Object { -not $_.Contains('too red') } |
Set-Content $_
}
Or will it be better to create a copy first using New-Item then process the other jobs?
It's going to be something very simple I know! And will most definitely kick myself once corrected.
Thanks in advance
Looks like what you want is something like this:
Get-ChildItem -Path 'C:\Temp' -File -Recurse | ForEach-Object {
$newFile = Join-Path -Path $_.DirectoryName -ChildPath ('{0}Ripe{1}' -f $_.BaseName, $_.Extension)
$newContent = Get-Content $_.FullName |
Where-Object { $_ -like '*red*' -and $_ -notlike '*too red*' } |
ForEach-Object { $_ -replace "[/()]", ":" }
$newContent | Set-Content -Path $newFile
}
To complement Theo's helpful answer - which is probably the most straightforward in your case - with a streaming, single-pipeline solution that showcases two advanced techniques:
the common -PipelineVariable (-pv) parameter, which allows you to store a cmdlet's current pipeline output object in a self-chosen variable that can be referenced later in a script block in a later pipeline segment.
delay-bind script blocks, which allow you to use a script block to dynamically determine a parameter value, typically based on the pipeline input object at hand; in this case, the pipeline variable is used.
# Create sample files
'red1' > t1.txt
'red2' > t2.txt
Get-ChildItem -PipelineVariable file t?.txt | # note `-PipelineVariable file`
Get-Content | # read file line by line
Where-Object { $_.Contains('red') } | # sample filter
ForEach-Object { $_ -replace 'e', '3' } | # sample transformation
Set-Content -LiteralPath { # delay-bind script block
# Determine the output file name based on pipelinve variable $file
'{0}Ripe{1}' -f $file.BaseName, $file.Extension
}
This results in files t1Ripe.txt and t2Ripe.txt, containing r3d1 and r3d2, respectively.

Powershell copy files based on last modified date & rename them afterwards

i´ve got little bit stucked with my powershell script.
I would like to run through multiple folders, grab files based on their last modified date and copy them to a new location.
There i have to rename them to a specific convention based on it´s original Filename.
What i wrote only runs through the first part and copy files successfully but not rename them afterwards. Of course when i run the script a second time it renames the files...
File convention is:
120_00001_000_002222_202201_20220124_121833_Formular - Copy.pdf
result should be
2222_120_Memory 01-2022_012022.pdf
this is what i got already
$path = "G:\Temp"
$Target = "K:\Local"
$Max_days = "-60" #Max Days past
$Curr_date = Get-Date
$files = get-childitem $Target *.pdf
Get-ChildItem -Path $path -Recurse -Filter 120_*.pdf |
Where-Object {
$_.LastWriteTime `
-gt (Get-Date $Curr_date.AddDays($Max_days)) `
} | ForEach-Object { $_ | Copy-Item -Destination $Target -Force -PassThru }
foreach($pdf in $files)
{
$split = $pdf.name -replace ".pdf" -split "_"
$newname = "$($split[3].TrimStart("0"))_$($split[0])_$("Memory") $($split[4].Substring($split[4].Length - 2, 2))-$($split[5].Substring(0,4))_$($split[4].Substring($split[4].Length - 2, 2))$($split[5].Substring(0,4))$($pdf.Extension)"
write-verbose "Original: $($pdf.name)" -verbose
write-verbose "NewName: $($newname)" -verbose
Rename-Item $pdf.FullName -NewName $newname -verbose
}
Thanks in adavnced
Edited the Question to more precision.
As commented, you could do this in one loop and rename the file while copying.
Try below:
$path = 'G:\Temp'
$Target = 'K:\Local'
$Max_days = -60 # Max Days in the past
$refDate = (Get-Date).AddDays($Max_days).Date # set to midnight
# get the files of interest
Get-ChildItem -Path $path -Recurse -Filter '120_*_*_*_*_*_*_*.pdf' -File |
Where-Object { $_.LastWriteTime -gt $refDate } |
ForEach-Object {
# rename the file to match the new file naming convention
$split = $_.BaseName -split "_"
# just for clarity, using this example:
# '120_00001_000_002222_202201_20220124_121833_Formular - Copy.pdf'
# $split[0] --> 120 used unchanged
# $split[1] --> 00001 unused
# $split[2] --> 000 unused
# $split[3] --> 002222 used without leading zeros
# $split[4] --> 202201 used, only the last two digits (month)
# $split[5] --> 20220124 used, only the first four digits (year)
# $split[6] --> 121833 unused
# $split[7] --> Formular - Copy unused
# these elements are used more than once, so for convenience store in separate variables
$month = $split[4].Substring($split[4].Length - 2, 2)
$year = $split[5].Substring(0,4)
# construct the new file name
$newName = '{0}_{1}_Memory {2}-{3}_{2}{3}{4}' -f $split[3].TrimStart("0"),
$split[0],
$month,
$year,
$_.Extension
# construct the complete target path and filename
$targetFile = Join-Path -Path $Target -ChildPath $newName
# now copy the file with a new name to the target folder
$_ | Copy-Item -Destination $targetFile -Force
}
I've used the -f Format operator to construct the new filename, because I believe this makes the code easier to read.
I did not take into consideration that naming collisions might occur (file with that new name already in the target folder).
If that can happen, you need to tell us what strategy to use.
Perhaps append an index number to the file in brackets like Windows does?

Advance file search with powershell

I'm fairly new to Powershell and programming in general. I want to search files using Powershell having multiple conditions. I have managed to write this code
$Drives = Get-PSDrive -PSProvider 'FileSystem'
$Filename= 'Result'
$IncludeExt= '*csv,*docx'
$StartDate= '11/1/20'
$EndDate= '1/26/21'
Get-ChildItem -Path $Drives.Root -Recurse |Where-Object {$IncludeExt -match $_.Extension} | Where-Object { $_.BaseName -match $Filename} | Where-Object {$_.lastwritetime -ge $StartDate -AND $_.lastwritetime -le $EndDate} |
foreach{
$Item = $_.Basename
$Path = $_.FullName
$Type = $_.Extension
$Age = $_.CreationTime
$Path | Select-Object `
#{n="Name";e={$Item}},`
#{n="Created";e={$Age}},`
#{n="filePath";e={$Path}},`
#{n="Folder/File";e={if($Folder){"Folder"}else{$Type}}}`
}| Export-Csv D:\FFNew.csv -NoTypeInformation
This works well when the all variables are mentioned. But how do I get this to work when
Case1: If $Filename is empty then it gives all the files with the mentioned extensions and files modified in Range of dates
Case2: If $IncludeExt is left empty then it gives all files with the $Filename mentioned, currently it gives only the folders and files modified in Range of dates
Case 3: If $Filename and $IncludeExt is left empty it gives all the files modified between the $StartDate and $EndDate
Pranay,
[EDITED]
Ok, here's the revised (exact) script with notes and sample output. Note: you'll have to change the items that are specific to my machine!
$Drives = Get-PSDrive -PSProvider 'FileSystem'
$Filename = "*" #for all or "*partial name*"
$IncludeExt = $Null #for no ext. or "*.csv","*.docx",etc...
$StartDate = '01/1/2020' #to ignore this use 1/1/1920
#For latest date use below otherwise specify date.
$EndDate = (Get-Date).ToShortDateString()
#Note: below uses only 3rd drive in the array remove [2] for all.
$GCIArgs = #{Path = $Drives[2].Root
Recurse = $True
}
If ($Null -ne $IncludeExt) {
$GCIArgs.Add("Include",$IncludeExt)
}
Get-ChildItem #GCIArgs |
Where-Object {($_.BaseName -Like $Filename) -and
($_.lastwritetime -ge $StartDate) -and
($_.lastwritetime -le $EndDate) } |
foreach{
$Item = $_.Basename
$Path = $_.FullName
$Type = $_.Extension
$Type = & {if($_.PSIsContainer){"Folder"}else{$_.Extension}}
$Age = $_.CreationTime
$Path | Select-Object #{n="Name" ;e={$Item}},
#{n="Created" ;e={$Age}} ,
#{n="filePath" ;e={$Path}},
#{n="Folder/File";e={$Type}}
} | Export-Csv -LiteralPath 'G:\BEKDocs\FFNew.csv' -NoTypeInformation
Notes:
$IncludeExt is specified as $Null if it is not used and if used the list is like this ".csv",".docx"
$Filename is specified as "*" for all filenames. Also changed the test from -match to -like so partial filenames should include *, e.g. "partial name".
Notice I changed the location of the check for Extensions to use the -Include parameter of the Get-ChildItem vs checking in the Where-Object.
Changed the piping of data to successive Where-Object clauses and replaced with -and operator, same effect and more efficient.
Changed the test for Directories to use the PSIsContainer property, couldn't see where you were getting the value for $Folder.
Removed the continuation characters from the Select-Object as the comma serves that purpose and is cleaner.
Sample output on Single drive (per code shown above) with some lines hidden for space considerations but notice the last line number.
Sample output on all drives (code edited as per comment in code), again lines hidden for space but showing multiple drives and final line number.
HTH

Powershell rename files whilst maintaining the original file order

I've got a set of files in the following format
I'd like to change the file names to the following format
I've used the following code:
`$i = 21
dir | ForEach-Object {$_ | Rename-Item -NewName ('00816-101998-XX-A-2-50-{0:D3} Detalj.{1}' -f $i++, $_.Extension)}`
This code starts renaming with file name A.10.11.11
How do I get it to start from the first file name i.e. 8.A.10.11.8?
Thank you!
Looking at your question, I think this is what you intend to do.
The list of current filenames should be sorted by the integer value the filenames start with.
Next you want to rename them using a counter that starts with value 21
To do this, you can use a ForEach-Object loop, but the -NewName parameter of Rename-Item can also contain a scriptblock containing the action to perform.
$i = 21
(Get-ChildItem -Path 'D:\Test' -Filter '*.pdf' -File) |
Sort-Object #{Expression = {[int]($_.Name -split ' ')[0]}} |
Rename-Item -NewName { '00816-101998-XX-A-2-50-{0:D3} Detalj{1}' -f $script:i++, $_.Extension }
P.S. The image shows the filename starts with a number followed by a space, while in the question you give an example where the number is followed by a dot .. If that is the case in you file names, change [int]($_.Name -split ' ')[0] into [int]($_.Name -split '\.')[0]
1. We need to address and increment the value of the counter $i using the $script:i scoping syntax, otherwise $i does not exist in the NewName scriptblock
2. To avoid renaming files twice, enclose the Get-ChildItem part in the code in between brackets
param(
$directory = "d:\tmp",
[regex]$rx = "(?<=8.A.10.11.)\d+",
[string]$newPrefix = "00816-101998-XX-A-2-50-"
)
<#
8..20 | ForEach-Object {
$filePath = Join-Path $directory -ChildPath "8.A.10.11.$_.pdf"
Out-File -InputObject "some text" -FilePath $filePath -Encoding default
}
#>
Get-ChildItem $directory | ForEach-Object{
Rename-Item $_.FullName -NewName "$newPrefix$((($rx.Matches($_.BaseName)).value).padleft(3,'0'))$($_.Extension)"
}

Powershell and last modified date

I am working in a windows environment.
I have a project that requires a short script to determine if a file with a modified date of today exists in a folder. If the file exists, it should copy it, if a file does not exist, it should return an error code.
I prefer to not use 3rd party apps. I am considering powershell.
I can pull a list to visually determine if the file exists, but I am having trouble batching to return an error if the count is zero.
Get-ChildItem -Path C:\temp\ftp\archive -Recurse | Where-Object { $_.lastwritetime.month -eq 3 -AND $_.lastwritetime.year -eq 2013 -AND $_.lastwritetime.day -eq 21}
Any help is greatly appreciated!
You can compare the current date against the date part only of each file LastWriteTime short date:
Get-ChildItem -Path C:\temp\ftp\archive -Recurse | Where-Object {
$_.LastWriteTime.ToShortDateString() -eq (Get-Date).ToShortDateString()
}
Get-ChildItem $path -r | % {if((!($_.psiscontianer))-and(Get-Date $_.LastWriteTime -Uformat %D)-eq(Get-Date -UFormat %D)){$_.FullName}else{Write-Warning 'No from Today'}}
F.Y.I. when doing large jobs, like if you'll be going through TB of files, use a foreach-object. It's faster then Where-Object. This method processes the objects collected in the array directly when available and doesn't wait until all objects are collected.
In summary, there always a lot of different ways to achieve the same result in PowerShell. I advocate using what is easiest for you to remember. At the same time, PowerShell can provide some big performance differences between the approaches – and it pays to know more!
You can still make the line a little more efficient by calculating the date
$date = (Get-Date -UFormat %D)
Get-ChildItem $path -r | % {if((!($_.psiscontianer))-and(Get-Date $_.LastWriteTime -Uformat %D)-eq$date){$_.FullName}else{Write-Warning 'No from Today'}}
I was able to use the following script:
$Date = Get-Date
$Date = $Date.adddays(-1)
$Date2Str = $Date.ToString("yyyMMdd")
$Files = gci "C:\\Temp\\FTP\\Archive"
ForEach ($File in $Files){
$FileDate = $File.LastWriteTime
$CTDate2Str = $FileDate.ToString("yyyyMMdd")
if ($CTDate2Str -eq $Date2Str) {Copy-Item $File.Fullname "C:\\Temp\\FTP"; exit}
}
Throw "No file was found to process"
To test if there are no files:
$out = Get-ChildItem -Path C:\temp\ftp\archive -Recurse | Where-Object {
$_.LastWriteTime.ToShortDateString() -eq (Get-Date).ToShortDateString()
};
if ($out.Count -gt 0)
//do something with your output
else
//sorry no file