I have customized one powershell code to zip files older than 7 days from a source folder to a subfolder and then delete the original files from source after zipping is complete. The code is working fine with inbuilt Compress-Archive and Remove-Item cmdlets with less volume of files, but takes more time and system memory for a large volume of files. So, I'm working on a solution using 7zip instead as it's faster.
Below script does zipping correctly but not following the condition of only files older than 7 days and deletes all the files from source folder. It should zip and delete only files older than 7 days.
I have tried all possible ways to troubleshoot but no luck. Can anybody suggest possible solution?
if (-not (test-path "$env:ProgramFiles\7-Zip\7z.exe")) {throw "$env:ProgramFiles\7-Zip\7z.exe needed"}
set-alias sz "$env:ProgramFiles\7-Zip\7z.exe"
$Date = Get-Date -format yyyy-MM-dd_HH-mm
$Source = "C:\Users\529817\New folder1\New folder_2\"
$Target = "C:\Users\529817\New folder1\New folder_2\ARCHIVE\"
Get-ChildItem -path $Source | sz a -mx=9 -sdel $Target\$Date.7z $Source
There are several problems here. The first is that 7-Zip doesn't accept a list of files as a pipe, furthermore even if it did your GCI is selecting every file and not selecting by date. The reason that it works at all is that you are passing the source folder as a parameter to 7-Zip.
7-Zip accepts the list of files to zip as a command line argument:
Usage: 7z <command> [<switches>...] <archive_name> [<file_names>...] [#listfile]
And you can select the files you want by filter the output from GCI by LastWriteTime.
Try changing your last line to this
sz a -mx=9 -sdel $Target\$Date.7z (gci -Path $Source |? LastWriteTime -lt (Get-Date).AddDays(-7) | select -expandproperty FullName)
If you have hundreds of files and long paths then you may run into problems with the length of the command line in which case you might do this instead:
gci -Path $Source |? LastWriteTime -lt (Get-Date).AddDays(-7) |% { sz a -mx=9 -sdel $Target\$Date.7z $_.FullName }
Consider a temporary file with a list of those files which need to be compressed:-
$tmp = "$($(New-Guid).guid).tmp"
set-content $tmp (gci -Path $Source |? LastWriteTime -lt (Get-Date).AddDays(-7)).FullName
sz a -mmt=8 out.7z #$tmp
Remove-Item $tmp
Also looking at the parameters to 7-Zip: -mx=9 will be slowest for potentially a small size gain. Perhaps leave that parameter out and take the default and consider adding -mmt=8 to use multiple threads.
Related
I've written code to zip files older than 7 days from a source folder to a subfolder and then delete the original files. My code works best with Compress-Archive and Remove-Item cmdlets with fewer files, but takes more time and system memory for a large volume of files.
So, I'm working on a solution using 7zip instead as it's faster.
Below code does zipping correctly but not limit itself to files older than 7 days and deletes all the files from source folder. It should zip and delete only files older than 7 days.
Is there anything wrong with the code.
if (-not (test-path "$env:ProgramFiles\7-Zip\7z.exe")) {throw "$env:ProgramFiles\7-Zip\7z.exe needed"}
set-alias 7z "$env:ProgramFiles\7-Zip\7z.exe"
$Days = "7"
$Date = Get-Date -format yyyy-MM-dd_HH-mm
$limit = (Get-Date).AddDays(-$Days)
$filePath = "C:\Users\529817\New folder1\New folder_2"
Where LastWriteTime -lt $limit | 7z a -t7z -sdel "C:\Users\529817\New folder1\New folder_2\ARCHIVE\$Date.7z" "$filePath"
I don't think you are running the 7zip command correctly. You are simply telling it to add all the files from the directory $filepath to the archive then delete all the files. That and I have serious doubts that 7zip can take pipeline input as your sample suggests.
Look at the examples from 7Zip cmdline help:
7z a archive1.zip subdir\
Adds all files and subfolders from folder subdir to archive archive1.zip. The filenames in archive will contain subdir\ prefix.
7z a archive2.zip .\subdir\*
Adds all files and subfolders from folder subdir to archive archive2.zip. The filenames in archive will not contain subdir\ prefix.
I'd have to download 7Zip to test but I think you need a loop to process the files you isolated with the Where clause. It might look something like:
if (-not (test-path "$env:ProgramFiles\7-Zip\7z.exe")) {throw "$env:ProgramFiles\7-Zip\7z.exe needed"}
set-alias 7z "$env:ProgramFiles\7-Zip\7z.exe"
$Days = "7"
$Date = Get-Date -format yyyy-MM-dd_HH-mm
$limit = (Get-Date).AddDays(-$Days)
$filePath = "C:\Users\529817\New folder1\New folder_2"
Get-ChildItem $filePath |
Where-Object{ $_.LastWriteTime -lt $limit } |
ForEach-Object{
7z a -t7z -sdel "C:\Users\529817\New folder1\New folder_2\ARCHIVE\$Date.7z" $_.FullName
}
Note: At least in your sample you are missing the Get-ChildItem command. I don't think you need to reference the .Date property from the [DateTime] object returned by the .AddDays() method unless you want the boundary to be midnight of that date. Otherwise .AddDays() will return a [DateTime] naturally.
Ultimately, I need a solid PowerShell script that will take a folder with several hundred video files, import the existing file names into the program, lookup the new file name in a CSV, and rename it. The old filename is simply (ie. File1.mp4, File2.mp4, etc.) I would like to appended a date to the front of the file in the format of (YYYY-MM-DD).
For testing, I created a folder on my desktop with (10) text files, each with a unique file name.
My CSV file appears as follows:
Image of CSV
The "newfilename" column, was created by using the Concatenate command in Excel.
`(=CONCATENATE(TEXT(A2, "yyyy-mm-dd")," ", B2)`
As much as I would just like PowerShell to handle everything, I feel using Excel for most of this might be the best way.
In my testing, everything was in one folder. However, at work, I will have video files on one drive, and the script will have to be in a folder on my desktop. Because I am in a corporate network, I need a special batch file to run my scripts, which is nothing new. I just modify the script name, and away it goes!
So what commands do I need to do in order to have the script separate from the video files AND the CSV file?
Here is the code that I have so far. Everything works when it's in one folder.
PS C:\Users\ceran\Desktop\Rename Project> Import-Csv -Path .\MyFileList.csv | ForEach-Object {
>> $Src = Join-Path -Path $TargetDir -ChildPath $_.filename
>> $Dst = Join-Path -Path $TargetDir -ChildPath $_.newfilename
>> Rename-Item -Path $Src -NewName $Dst
>> }
Thanks in advance for the help!
Chris
I'm not sure what the date column is in your Excel file and if you want to rename all files in the folder, but if that is the case, you don't need a csv file at all and can do this:
$sourceFolder = 'X:\Path\to\the\video\files' # change this to the real path
Get-ChildItem -Path $sourceFolder -Filter '*.mp4' -File | # iterate through the files in the folder
Where-Object {$_.Name -notmatch '^\d{4}-\d{2}-\d{2}'} | # don't rename files that already start with the date
Rename-Item -NewName { '{0:yyyy-MM-dd} {1}' -f $_.LastWriteTime, $_.Name } -WhatIf
This uses parameter -Filter '*.mp4', to get only files with an .mp4 extension. For the files in your testfolder (Desktop\Rename Project), change this to -Filter '*.txt'.
If you want all files renamed, no matter what the extension, simply remove the Filter from the cmdlet.
Because of the -WhatIf switch, no file is actually renamed and the code just shows in the console what would happen. Once satisfied that this is OK, remove the -WhatIf
Hope that helps.
$targetdir="C:\path\to\where\our\file\directory\is"
$pathtocsv="c:\path\to\csv.csv"
Import-Csv -Path $pathtocsv | ForEach-Object {
$Src = Join-Path -Path $TargetDir -ChildPath $_.filename
$Dst = Join-Path -Path $TargetDir -ChildPath $_.newfilename
Rename-Item -Path $Src -NewName $Dst
}
Why would this not work in any situation?
By the way, if the csv had the columns path and newname, it could be piped directly to rename-item:
path,newname
file.txt,file2.txt
import-csv ren.csv | Rename-Item -whatif
What if: Performing the operation "Rename File" on target "Item: /Users/js/foo/file.txt Destination: /Users/js/foo/file2.txt".
This powershell command works perfectly to copy and extract a zip file across two directories:
$shell = New-Object -COM Shell.Application
$target = $shell.NameSpace('D:\destination\')
$zip = $shell.NameSpace('D:\source\version_*.zip')
$target.CopyHere($zip.Items(), 16)
However I am struggling with a modification to make it select only the latest zip file from the source.
Get the zip file with the most recent modification date in a given directory:
$source = "C:\temp"
$destination = "C:\temp\output"
$zipFile = Get-ChildItem -Path $source -Filter "*.zip" |
Sort-Object LastWriteTime -Descending |
Select-Object -First 1
Expand-Archive -Path $zipFile.FullName -DestinationPath $destination
This works by searching for all zip files, sorting them by the modified date descending and then grabbing the "first" one (according to the defined sort order).
I've also used the Expand-Archive command to extract the zip to a specified destination. If you need to copy the zip then that's easy enough using the Copy-Item cmdlet first.
As some commenters have pointed out: Expand-Archive was introduced in version 5 of PowerShell.
However the logic of getting the "latest" file is unchanged and can easily be plumbed in to your existing script:
$zip = $shell.NameSpace($zipFile.FullName)
So I have danced with this off and on throughout the day and the timeless phrase "There's more than one way to skin a cat" keeps coming to mind so I decided to take to the community.
Scenario:
Source folder "C:\Updates" has 100 files of various extensions. All need to be copied to the sub-folders only of "C:\Prod\" overwriting any duplicates that it may find.
The Caveats:
The sub-folder names (destinations) in "C:\Prod" are quite dynamic and change frequently.
A naming convention is used to determine which sub-folders in the destination need to be excluded when the source files are being copied (to retain the original versions). For ease of explanation lets say any folder names starting with "!stop" should be excluded from the copy process. (!stop* if wildcards considered)
So, here I am wanting the input of those greater than I to tackle this in PS if I'm lucky. I've tinkered with Copy-Item and xcopy today so I'm excited to hear other's input.
Thanks!
-Chris
Give this a shot:
Get-ChildItem -Path C:\Prod -Exclude !stop* -Directory `
| ForEach-Object { Copy-Item -Path C:\Updates\* -Destination $_ -Force }
This grabs each folder (the -Directory switch ensures we only grab folders) in C:\Prod that does not match the filter and pipes it to the ForEach-Object command where we are running the Copy-Item command to copy the files to the directory.
The -Directory switch is not available in every version of PowerShell; I do not know which version it was introduced in off the top of my head. If you have an older version of PowerShell that does not support -Directory then you can use this script:
Get-ChildItem -Path C:\Prod -Exclude !stop* `
| Where-Object { $_.PSIsContainer } `
| ForEach-Object { Copy-Item -Path C:\Updates\* -Destination $_ -Force }
To select only sub folders which do not begin with "!stop" do this
$Source = "C:\Updates\*"
$Dest = "C:\Prod"
$Stop = "^!stop"
$Destinations = GCI -Path $Dest |?{$_.PSIsContainer -and $_.Name -notmatch $Stop }
ForEach ($Destination in $Destinations) {
Copy-Item -Path $Source -Destination $Destination.FullName -Force
}
Edited Now copies all files from Update to subs of Source not beginning with "!stop" The -whatif switch shows what would happen, to arm the script remove the -whatif.
Edit2 Streamlined the script. If also Sub/sub-folders of C:\Prod shall receive copies include a -rec option to the gci just in front of he pipe.
I am trying to write a PowerShell script that will copy a subset of files from a source folder and place them into a target folder. I've been playing with "copy-item" and "remove-item" for half a day and cannot get the desired or consistent results.
For example, when I run the following cmdlet multiple times, the files end up in different locations?!?!:
copy-item -Path $sourcePath -Destination $destinationPath -Include *.dll -Container -Force -Recurse
I've been trying every combination of options and commands I can think of but can't find the right solution. Since I'm sure that I'm not doing anything atypical, I'm hoping someone can ease my pain and provide me with the proper syntax to use.
The source folder will contain a large number of files with various extensions. For example, all of the following are possible:
.dll
.dll.config
.exe
.exe.config
.lastcodeanalysisissucceeded
.pdb
.Test.dll
.vshost.exe
.xml
and so on
The script needs to only copy .exe, .dll and .exe.config files excluding any .test.dll and .vshost.exe files. I also need the script to create the target folders if they don't already exist.
Any help getting me going is appreciated.
try:
$source = "C:\a\*"
$dest = "C:\b"
dir $source -include *.exe,*.dll,*.exe.config -exclude *.test.dll,*.vshost.exe -Recurse |
% {
$sp = $_.fullName.replace($sourcePath.replace('\*',''), $destPath)
if (!(Test-Path -path (split-path $sp)))
{
New-Item (split-path $sp) -Type Directory
}
copy-item $_.fullname $sp -force
}
As long as the files are in one directory, the following should work fine. It might be a bit more verbose than needed, but it should be a good starting point.
$sourcePath = "c:\sourcePath"
$destPath = "c:\destPath"
$items = Get-ChildItem $sourcePath | Where-Object {($_.FullName -like "*.exe") -or ($_.FullName -like "*.exe.config") -or ($_.FullName -like "*.dll")}
$items | % {
Copy-Item $_.Fullname ($_.FullName.Replace($sourcePath,$destPath))
}