Zipping using powershell - powershell

I've written code to zip files older than 7 days from a source folder to a subfolder and then delete the original files. My code works best with Compress-Archive and Remove-Item cmdlets with fewer files, but takes more time and system memory for a large volume of files.
So, I'm working on a solution using 7zip instead as it's faster.
Below code does zipping correctly but not limit itself to files older than 7 days and deletes all the files from source folder. It should zip and delete only files older than 7 days.
Is there anything wrong with the code.
if (-not (test-path "$env:ProgramFiles\7-Zip\7z.exe")) {throw "$env:ProgramFiles\7-Zip\7z.exe needed"}
set-alias 7z "$env:ProgramFiles\7-Zip\7z.exe"
$Days = "7"
$Date = Get-Date -format yyyy-MM-dd_HH-mm
$limit = (Get-Date).AddDays(-$Days)
$filePath = "C:\Users\529817\New folder1\New folder_2"
Where LastWriteTime -lt $limit | 7z a -t7z -sdel "C:\Users\529817\New folder1\New folder_2\ARCHIVE\$Date.7z" "$filePath"

I don't think you are running the 7zip command correctly. You are simply telling it to add all the files from the directory $filepath to the archive then delete all the files. That and I have serious doubts that 7zip can take pipeline input as your sample suggests.
Look at the examples from 7Zip cmdline help:
7z a archive1.zip subdir\
Adds all files and subfolders from folder subdir to archive archive1.zip. The filenames in archive will contain subdir\ prefix.
7z a archive2.zip .\subdir\*
Adds all files and subfolders from folder subdir to archive archive2.zip. The filenames in archive will not contain subdir\ prefix.
I'd have to download 7Zip to test but I think you need a loop to process the files you isolated with the Where clause. It might look something like:
if (-not (test-path "$env:ProgramFiles\7-Zip\7z.exe")) {throw "$env:ProgramFiles\7-Zip\7z.exe needed"}
set-alias 7z "$env:ProgramFiles\7-Zip\7z.exe"
$Days = "7"
$Date = Get-Date -format yyyy-MM-dd_HH-mm
$limit = (Get-Date).AddDays(-$Days)
$filePath = "C:\Users\529817\New folder1\New folder_2"
Get-ChildItem $filePath |
Where-Object{ $_.LastWriteTime -lt $limit } |
ForEach-Object{
7z a -t7z -sdel "C:\Users\529817\New folder1\New folder_2\ARCHIVE\$Date.7z" $_.FullName
}
Note: At least in your sample you are missing the Get-ChildItem command. I don't think you need to reference the .Date property from the [DateTime] object returned by the .AddDays() method unless you want the boundary to be midnight of that date. Otherwise .AddDays() will return a [DateTime] naturally.

Related

using 7zip to zip in powershell 5.0

I have customized one powershell code to zip files older than 7 days from a source folder to a subfolder and then delete the original files from source after zipping is complete. The code is working fine with inbuilt Compress-Archive and Remove-Item cmdlets with less volume of files, but takes more time and system memory for a large volume of files. So, I'm working on a solution using 7zip instead as it's faster.
Below script does zipping correctly but not following the condition of only files older than 7 days and deletes all the files from source folder. It should zip and delete only files older than 7 days.
I have tried all possible ways to troubleshoot but no luck. Can anybody suggest possible solution?
if (-not (test-path "$env:ProgramFiles\7-Zip\7z.exe")) {throw "$env:ProgramFiles\7-Zip\7z.exe needed"}
set-alias sz "$env:ProgramFiles\7-Zip\7z.exe"
$Date = Get-Date -format yyyy-MM-dd_HH-mm
$Source = "C:\Users\529817\New folder1\New folder_2\"
$Target = "C:\Users\529817\New folder1\New folder_2\ARCHIVE\"
Get-ChildItem -path $Source | sz a -mx=9 -sdel $Target\$Date.7z $Source
There are several problems here. The first is that 7-Zip doesn't accept a list of files as a pipe, furthermore even if it did your GCI is selecting every file and not selecting by date. The reason that it works at all is that you are passing the source folder as a parameter to 7-Zip.
7-Zip accepts the list of files to zip as a command line argument:
Usage: 7z <command> [<switches>...] <archive_name> [<file_names>...] [#listfile]
And you can select the files you want by filter the output from GCI by LastWriteTime.
Try changing your last line to this
sz a -mx=9 -sdel $Target\$Date.7z (gci -Path $Source |? LastWriteTime -lt (Get-Date).AddDays(-7) | select -expandproperty FullName)
If you have hundreds of files and long paths then you may run into problems with the length of the command line in which case you might do this instead:
gci -Path $Source |? LastWriteTime -lt (Get-Date).AddDays(-7) |% { sz a -mx=9 -sdel $Target\$Date.7z $_.FullName }
Consider a temporary file with a list of those files which need to be compressed:-
$tmp = "$($(New-Guid).guid).tmp"
set-content $tmp (gci -Path $Source |? LastWriteTime -lt (Get-Date).AddDays(-7)).FullName
sz a -mmt=8 out.7z #$tmp
Remove-Item $tmp
Also looking at the parameters to 7-Zip: -mx=9 will be slowest for potentially a small size gain. Perhaps leave that parameter out and take the default and consider adding -mmt=8 to use multiple threads.

Powershell Script to clean up logs on a Monthly Basis

Powershell newbie here again with another question.
We currently have a log folder accumulating text files. My supervisor would like for me to create a script where only the current month's logs are visible. All previous months should be moved to an archive folder. He would like for me to create a powershell script we can run once a month to achieve this.
For example, our log folder should only have logs from January 2021. Anything older than January 2021 should be archived. Once February 1, 2021 hits, all the logs from January 2021 should be moved to the archive folder, and so on.
How can I achieve a script that looks at the folder and only keeps the logs for the current month? Any guidance, resources, videos, etc are greatly appreciated! I've been scouring the internet for resources, but I haven't quite found anything that suits my needs.
Update: Was able to find a wonderful script here: PowerShell: Sort and Move Files by Date to month and year provided by Thomas Maurer (all credit to him!)
# Get the files which should be moved, without folders
$files = Get-ChildItem 'C:\Users\testuser\Desktop\log' -Recurse | where {!$_.PsIsContainer}
# List Files which will be moved
$files
# Target Filder where files should be moved to. The script will automatically create a folder for the year and month.
$targetPath = 'C:\Users\testuser\Desktop\log\archive'
foreach ($file in $files)
{
# Get year and Month of the file
# I used LastWriteTime since this are synced files and the creation day will be the date when it was synced
$year = $file.LastWriteTime.Year.ToString()
$month = $file.LastWriteTime.Month.ToString()
# Out FileName, year and month
$file.Name
$year
$month
# Set Directory Path
$Directory = $targetPath + "\" + $year + "\" + $month
# Create directory if it doesn't exsist
if (!(Test-Path $Directory))
{
New-Item $directory -type directory
}
# Move File to new location
$file | Move-Item -Destination $Directory
}
What I would like to achieve now: This script works great, but I am trying to tinker with it so I can move everything EXCEPT the current month. I'm still researching and investigating, so I will make sure to update my post if I am able to figure out this missing piece for me. Thank you all for your help!
One approach to leave the files that were last modified in this month is to use a small helper function that formats the LastWriteTime date into a string yyyy\MM.
function Format-YearMonth ([datetime]$date) {
# simply output a string like "2021\01"
return '{0:yyyy\\MM}' -f $date
}
$sourcePath = 'C:\Users\testuser\Desktop\log'
$targetPath = 'C:\Users\testuser\Desktop\log\archive'
$thisMonth = Format-YearMonth (Get-Date)
# Get the files which should be moved, without folders
# this can be more efficient if all files have the same extension on which
# you can use -Filter '*.log' for instance.
Get-ChildItem -Path $sourcePath -File -Recurse |
# filter out the files that have a LastWriteTime for this year and month
Where-Object {(Format-YearMonth $_.LastWriteTime) -ne $thisMonth } |
ForEach-Object {
# Set destination Path
$Directory = Join-Path -Path $targetPath -ChildPath (Format-YearMonth $_.LastWriteTime)
# Create directory if it doesn't exsist
if (!(Test-Path $Directory)) {
$null = New-Item $Directory -type Directory
}
Write-Host "Moving file '$($_.FullName)' to '$Directory'"
# Move File to new location
$_ | Move-Item -Destination $Directory -Force
}

Powershell: Get-ChildItem performance to deal with bulk files

The scenario is in a remote server, a folder is shared for ppl to access the log files.
The log files will be kept for around 30 days before they got aged, and each day, around 1000 log files will be generated.
For problem analysis, I need to copy log files to my own machine, according to the file timestamp.
My previous strategy is:
Use dir /OD command to get the list of the files, to my local PC, into a file
Open the file, find the timestamp, get the list of the files I need to copy
Use copy command to copy the actual log files
It works but needs some manual work, ie step2 I use notepad++ and regular expression to filter the timestamp
I tried to use powershell as:
Get-ChildItem -Path $remotedir | where-object {$_.lastwritetime -gt $starttime -and $_lastwritetime -lt $endtime } |foreach {copy-item $_.fullname -destination .\}
However using this approach it took hours and hours and no file has been copied, while compared with the dir solution it took around 7-8 minutes to generate the list of the file than copy itself took sometime but not hours
I guess most of the time spent on the filter file. I'm not quite sure why the get-childitem's performance is so poor.
Can you please advise if there's anything i can change?
Thanks
For directories with a lot of files, Get-ChildItem is too slow. It looks like most of the time is spent enumerating the directory, then filtering through 'where', then copying each file.
Use .net directly, particularly [io.directoryinfo] with the GetFileSystemInfos() method.
e.g.
$remotedir = [io.directoryinfo]'\\server\share'
$destination = '.\'
$filemask = '*.*'
$starttime = [datetime]'jan-21-2021 1:23pm'
$endtime = [datetime]'jan-21-2021 4:56pm'
$remotedir.GetFileSystemInfos($filemask, [System.IO.SearchOption]::TopDirectoryOnly) | % {
if ($_.lastwritetime -gt $starttime -and $_.lastwritetime -lt $endtime){
Copy-Item -Path $_.fullname -Destination $destination
}
}

Trying to copy files with certain a date from folders into another location with a mirrored folder structure

We're on sort of a file timecrunch. There are a bunch of files from 4/9/2019 that we need out of a bunch of directories. Every directory has ~1000 files. We want to copy all those files from 4/9/2019 to their respective folder structures, but only those files.
We used xcopy to copy the folder structure and have been manually copying the files we need. However, there are 1000 folders each with ~1000 files so it makes quite the hurdle. Tried passing on our arguments to copy-item, but it hasn't been giving me much luck.
I'm assuming even with my garbled code that there's some easy parameter or something I overlooked. We've been working through break and I've browsed through many topics and am turning to the community for help. I probably passed my exact problem on the way here.
For the record I don't code and use PowerShell infrequently.
#Gets all files that were last modified on 4/9/2019
(Get-ChildItem -Recurse | Where-Object {$_.LastWriteTime -gt (Get-Date -month 4 -day 9 -Format d) -and $_.LastWriteTime -lt (Get-Date -month 4 -day 10 -Format d)}) |
#Copies the files from source to destination
Copy-Item "C:\abc\" "D:\abc\"
}
When working with [datetime] type variables don't apply -format D this converts to a string,
instead strip the time component by appending .Date to both sides and test for equality.
As you are recursing the source into subfolders, you can't specify a flat target
$Source = "C:\abc\"
$Target = "D:\abc\"
$MyDate = (Get-Date -Month 4 -Day 9 -Year 2019).Date
Get-ChildItem -Path $Src -Recurse |
Where-Object {$_.LastWriteTime.Date -eq $MyDate} | ForEach-Object{
$TargetDir = $_.DirectoryName.replace($Source,$Target)
If (!(Test-Path $TargetDir)){New-Item $TargetDir -ItemType Directory|Out-Null}
$_ | Copy-Item -Destination $TargetDir
}
Untested, you might append the -WhatIf parameter to Copy-Item while testing.
Robocopy is for sure faster, once you've found the right parameters.

Copy files from one directory to another

I need to copy files from one directory to another location, based on the age of the file. I also need to keep the directory structure.
This code is working as far as only copying the files that meet the criteria, but it is NOT keeping the directory structure:
$ListDate = Get-Date "12/6/2013 11:08 AM"
$ActiveDate = $ListDate.AddYears(-7)
Get-ChildItem -path "T:\ProductionServices" -recurse | where-object {$_.lastwritetime -le $ActiveDate -and -not $_.psiscontainer} | Copy-item -destination "T:\TECH\CopyOfDeleteFile"
I've been struggling with this for over a week, and I've tried all the suggestions I have seen here and on the internet. I just need a little push to figure out what I am doing wrong.
This is one of those situations that is better for robocopy than PowerShell (or, use robocopy with PowerShell). Although with robocopy, you can't get resolution down to the minute (I think)
robocopy t:\Productionservices t:\tech\copyofdeletefile /E /MINAGE:20061206
Or, if you want to use it with PowerShell to do the date math:
$ListDate = Get-Date "12/6/2013 11:08 AM";
$ActiveDate = get-date $($ListDate.AddYears(-7)) -f "yyyyMMdd";
robocopy t:\Productionservices t:\tech\copyofdeletefile /E /MINAGE:$ActiveDate;
You can use the /COPY:DAT /DCOPY:T switches to preserve all attributes & timestamps as well.