I want to use Powershell to automate the:
1. compression of log files (.xml and .dat extensions) older than 7 days,
2. copy these compressed archives elsewhere and
3. then delete the raw log files from source.
I am using the following Powershell script which I pieced together from various resources.
function New-Zip
{
param([string]$zipfilename)
set-content $zipfilename ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
(dir $zipfilename).IsReadOnly = $false
}
function Add-Zip
{
param([string]$zipfilename)
if(-not (test-path($zipfilename)))
{
set-content $zipfilename ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
(dir $zipfilename).IsReadOnly = $false
}
$shellApplication = new-object -com shell.application
$zipPackage = $shellApplication.NameSpace($zipfilename)
foreach($file in $input)
{
$zipPackage.CopyHere($file.FullName)
Start-sleep -milliseconds 500
}
}
$targetFolder = 'C:\source'
$destinationFolder = 'D:\destination\'
$now = Get-Date
$days = 7
$lastWrite = $now.AddDays(-$days)
Get-ChildItem $targetFolder -Recurse | Where-Object { $_ -is [System.IO.FileInfo] } | ForEach-Object {
If ($_.LastWriteTime -lt $lastWrite)
{
$_ | New-Zip $($destinationFolder + $_.BaseName + ".zip")
$_ | Add-Zip $($destinationFolder + $_.BaseName + ".zip")
}
}
Get-ChildItem $targetFolder -Recurse -Include "*.dat", "*.xml" | WHERE {($_.CreationTime -le $(Get-Date).AddDays(-$days))} | Remove-Item -Force
This script does work reasonably well, as it archives only the files, and copies them on destination folder.
If I have a structure of C:\source\bigfolder\logfile.dat, the resulting zip file will not get the folder structure as I would like:
logfile.zip>bigfolder>logfile.dat
Instead, it just gets: logfile.zip>logfile.dat
Can someone help in figuring this out ?
To fine tune it even better, I would like if possible to build some logic, so the files are compressed only when a specific criteria is met.
The raw log files that I compress have a naming routine as following:
Folders:
emstg#12_list\randomstring.xml
Individual log files:
emstg#12_query_data.xml
emstg#12_events_cache.dat etc...
As you may see the start of these files is same with emstg#number.
How to implement a "name-detection" mechanism in script above ?
Thanks
you could zip a folder by using [System.IO.Compression]
I wrote this based on your script.
My idea is to copy the whole folder structure of the file you need to compress into a temp folder and then zip that temp folder.
For the name-detection, you just need another where-object (modify the code as you want)
function Zip
{
param(
[string]$source,
[string]$des
)
add-type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Compression.ZipFile]::CreateFromDirectory($source,$des,'Optimal',$true)
Start-sleep -s 1
}
$targetFolder = "C:\source"
$destinationFolder = "C:\destination"
$temp = "C:\temp"
$now = Get-Date
$days = 7
$lastWrite = $now.AddDays(-$days)
$i = 1
Get-ChildItem $targetFolder -Recurse | Where-Object { $_ -is [System.IO.FileInfo] } | Where-Object {$_ -like "*.*"} | ForEach-Object {
If ($_.LastWriteTime -lt $lastWrite) {
$halfDir = $_.DirectoryName.Trim($targetFolder)
$s = $temp + "\" + $i + "\" + $halfDir
$d = $destinationFolder + "\" + $_.BaseName + ".zip"
Copy-Item $_.DirectoryName -Destination $s
Copy-Item $_.FullName -Destination $s
Zip -source $s -des $d
$i++
}
}
Related
How do I rename the file inside the compressed file?
$timestamp = (Get-Date).AddMonths(-1).ToString('yyyy-MM')
$newFolderTimeStamp = (Get-Date).AddMonths(-1).ToString('MM - MMM-yyyy')
$sourceFile = "D:\Testing\1.txt","D:\Testing\2.txt","D:\Testing\3.txt","D:\Testing\4.txt","D:\Testing\5.txt"
#$sharedrive = "\\fs2\Backups\it\users.xlsx"
$newFolder = "D:\Testing\bin\$newFolderTimeStamp.zip"
######Compressed File
function Add-Zip
{
param([string]$zipfilename)
if(-NOT (test-path($zipfilename)))
{
#create an empty zip file
set-content $zipfilename ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
#make sure it is not set to ReadOnly
(dir $zipfilename).IsReadOnly = $false
}
$shellApplication = new-object -com shell.application
$zipPackage = $shellApplication.NameSpace($zipfilename)
foreach($file in $input)
{
$zipPackage.CopyHere($file.FullName)
Start-sleep -milliseconds 500
}
}
######Get file for today's date
Get-ChildItem $sourceFile -Force | Where-Object{$_.LastWriteTime -gt $date -and $_.extension -ne ".zip"} | add-Zip $newFolder
Tried , but only rename the zip file name.
Get-ChildItem $sourceFile -Force | Rename-Item -NewName {$_.Basename + '_' + $timestamp + $_.Extension } | Where-Object{$_.LastWriteTime -gt $date -and $_.extension -ne ".zip"} | add-Zip $newFolder
Expected result
06 - Jun-2019.zip
1_2019-06.txt
2_2019-06.txt
3_2019-06.txt
4_2019-06.txt
5_2019-06.txt
I would like to go inside every folder recursive in some location,
for example: C:\Logs and zip files to the same folder.
I need to skip every .zip file. I don't know how to make my script to do this:
function New-Zip
{
param([string]$zipfilename)
set-content $zipfilename ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
(dir $zipfilename).IsReadOnly = $false
}
function Add-Zip
{
param([string]$zipfilename)
if(-not (test-path($zipfilename)))
{
set-content $zipfilename ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
(dir $zipfilename).IsReadOnly = $false
}
$shellApplication = new-object -com shell.application
$zipPackage = $shellApplication.NameSpace($zipfilename)
foreach($file in $input)
{
$zipPackage.CopyHere($file.FullName)
Start-sleep -milliseconds 500
}
}
$targetFolder = 'C:\Logs'
$now = Get-Date -Hour 0 -Minute 00 -Second 00
$days = 5
$lastWrite = $now.AddDays(-$days)
Get-ChildItem $targetFolder -Recurse -Exclude *zip* | Where-Object { $_ -is [System.IO.FileInfo] } | ForEach-Object {
If ($_.LastWriteTime -lt $lastWrite)
{
$_ | New-Zip $($targetFolder + $_.Name + ".zip" )
$_ | Add-Zip $($targetFolder + $_.Name + ".zip" )
}
}
You are currently excluding any file or folder that contains the phrase 'zip' in its name as you have a wildcard either side of it:
Get-ChildItem $targetFolder -Recurse -Exclude *zip*
To exclude zip files you just need to update your Exclude statement so that it matches only the zip file extension:
Get-ChildItem $targetFolder -Recurse -Exclude *.zip
$source = "C:\folder\*.*"
$destination = "C:\folder3\test.zip"
$results = (Get-ChildItem -Path $source -Filter *.* | ? {
$_.LastWriteTime -gt (Get-Date).AddDays(-1))
}
Add-Type -assembly "system.io.compression.filesystem"
[io.compression.zipfile]::CreateFromDirectory($results, $destination)
Now i need to select last modified files and zip either by compress or 7zip any help ?
To zip files via Powershell, I usually install 7-zip on the system and then include this function in the script:
#7-ZIP FILE FUNCTION (must install 7-zip on system)
#-------------------------------------------------------------------------------------
#
function ZipFile7 ([string] $pZipFile, [string] $pFiles) {
$7zipExe = "$($env:programfiles)\7-Zip\7z.exe"
$7zArgs = #("a", "-tzip", $pZipFile, $pFiles, "-r")
& $7zipExe $7zArgs
}
#-------------------------------------------------------------------------------------
$pZipFile is the full path of the file you wish to create, and $pFiles is the path to the files you wish to zip (can include wildcards like C:\*.txt).
$source = "C:\folder\*.*"
$destination = "C:\folder3\test.zip"
$results = Get-ChildItem -Path $source | Where {$_.LastWriteTime -gt (Get-Date).AddDays(-1)} | Select -ExpandProperty "FullName"
set-content $destination ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
$Shell = New-Object -Com Shell.Application
$ZipFile = $Shell.Namespace($destination)
$Results | ForEach {
$Count = $ZipFile.Items().Count
$ZipFile.CopyHere($_)
While ($ZipFile.Items().Count -eq $Count) {Start-Sleep -Milliseconds 200} # Wait till the file is zipped
}
Note: you might want to improve the while/wait statement (e.g. set a timeout) as it might hang if anything goes wrong with copying the file to the zip folder.
I run a program in windows powershell which lists name of files which have been modified before 30 days. I need to compress these files in a zip format. What should be the code?
#List all the folders in G:\logfiles
$folders = (Get-ChildItem -Path "G:\logfiles" | Where-Object {$_.Attributes -eq "Directory"} | Select Fullname)
#looping all folders
Foreach ($folder in $folders)
{
$files = Get-ChildItem G:\logfiles\$folder | Where{$_.LastWriteTime -lt (Get-Date).AddDays(-30)
}
Foreach ($file in $files)
{
Move-Item $file G:\logfiles\$folder\$N # *To move files to a folder name $N*
}
I will refer you to powershell 5.0 and the new Compress-Archive cmdlet
Compress-Archive -DestinationPath $folderToZip
With the down function you can compress your files.
function zipFile($Source ,$DestZip ){
$folder = Get-Item -Path $Source
$ZipFileName = $DestZip + ".zip"
set-content $ZipFileName ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
# Wait for the zip file to be created.
while (!(Test-Path -PathType leaf -Path $ZipFileName))
{
Start-Sleep -Milliseconds 20
}
$ZipFile = (new-object -com shell.application).NameSpace($ZipFileName)
#BACKUP - COPY
$ZipFile.CopyHere($Source)
Write-Host "#Created zip file :" $DestZip
}
I was hoping someone can help me out with this script which is currently baffling me, I'm not very good at powershell ;)
What I'm trying to achieve is a script that will scan folders and subfolders of my source, looking for *.wav files older than 60 days and then move them to an archive destination, creating a folder structure of \year\month
This is what i've got
$SourceDir = "d:\test"
$DestinationDir = "e:\test"
$files = get-childitem $SourceDir *.wav
foreach ($file in $files)
{
$Directory = $DestinationDir + "" + $file.CreationTime.Date.ToString('yyyy') + "\" + $file.CreationTime.Date.ToString('MM-MMM')
if (!(Test-Path $Directory))
{
New-Item $directory -type directory
}
where-object {$_.LastWriteTime -lt (get-date).AddDays(-61)} | move-item $file.fullname $Directory
}
The script runs without error but doesn't move files :/
--
Following on from Keiths comments below, my script looks as so:
$SourceDir = "d:\test\"
$DestinationDir = "e:\test\"
$date = (get-date).AddDays(-91)
$files = get-childitem -literalpath $SourceDir *.wav
foreach ($file in $files)
{
$Directory = $DestinationDir + "" + $file.CreationTime.Date.ToString('yyyy') + "\" + $file.CreationTime.Date.ToString('MM-MMM')
if (!(Test-Path $Directory))
{
New-Item $directory -type directory
}
if ($file.LastWriteTime -lt $date) {
Write-Host $file
move-item -LiteralPath $file.fullname -Destination $Directory
}
}
And is working perfectly! Thanks Keith
Where-Object is never at the beginning of a pipeline. It requires input to populate the $_ variable you usually use to compare against. Replace that line with:
if ($file.LastWriteTime -lt (get-date).AddMonths(-3)) {
move-item $file.fullname $Directory
}
Although I would compute the date outside the loop:
$date = (get-date).AddMonths(-3)
And use that inside the loop:
...
if ($file.LastWriteTime -lt $date) {
Move-Item -LiteralPath $file.fullname -Destination $Directory
}
...