PowerShell - copying multiple files - powershell
I've written a PowerShell script to copy new files across to a shared folder on a server.
I am wondering if there is a way that after I get the list of new files in the sub folders, I can copy them over together - other than using for-each and copying them one at a time - so that I can add a progress bar.
something like this could be a starting point
# define source and destination folders
$source = 'C:\temp\music'
$dest = 'C:\temp\new'
# get all files in source (not empty directories)
$files = Get-ChildItem $source -Recurse -File
$index = 0
$total = $files.Count
$starttime = $lasttime = Get-Date
$results = $files | % {
$index++
$currtime = (Get-Date) - $starttime
$avg = $currtime.TotalSeconds / $index
$last = ((Get-Date) - $lasttime).TotalSeconds
$left = $total - $index
$WrPrgParam = #{
Activity = (
"Copying files $(Get-Date -f s)",
"Total: $($currtime -replace '\..*')",
"Avg: $('{0:N2}' -f $avg)",
"Last: $('{0:N2}' -f $last)",
"ETA: $('{0:N2}' -f ($avg * $left / 60))",
"min ($([string](Get-Date).AddSeconds($avg*$left) -replace '^.* '))"
) -join ' '
Status = "$index of $total ($left left) [$('{0:N2}' -f ($index / $total * 100))%]"
CurrentOperation = "File: $_"
PercentComplete = ($index/$total)*100
}
Write-Progress #WrPrgParam
$lasttime = Get-Date
# build destination path for this file
$destdir = Join-Path $dest $($(Split-Path $_.fullname) -replace [regex]::Escape($source))
# if it doesn't exist, create it
if (!(Test-Path $destdir)) {
$null = md $destdir
}
# if the file.txt already exists, rename it to file-1.txt and so on
$num = 1
$base = $_.basename
$ext = $_.extension
$newname = Join-Path $destdir "$base$ext"
while (Test-Path $newname) {
$newname = Join-Path $destdir "$base-$num$ext"
$num++
}
# log the source and destination files to the $results variable
Write-Output $([pscustomobject]#{
SourceFile = $_.fullname
DestFile = $newname
})
# finally, copy the file to its new location
copy $_.fullname $newname
}
# export a list of source files
$results | Export-Csv c:\temp\copylog.csv -NoTypeInformation
NOTE: that will show progress for total files regardless of size. ex: you have 2 files, one is 1 mb and the other is 50 mb. when the first file is copied, progress will be 50% because half the files are copied. if you want progress by total bytes, i highly recommend giving this function a try. just give it a source and destination. works when given a single file or a whole folder to copy
https://github.com/gangstanthony/PowerShell/blob/master/Copy-File.ps1
screenshot: http://i.imgur.com/hT8yoUm.jpg
Related
PowerShell: "Move-Item : Cannot create a file when that file already exists."
The code below has been wonderful so far for organising my hard-drives. I do face this error when I transfer large amounts of data: Move-Item : Cannot create a file when that file already exists. This happens when I move a file that is duplicate, is there a way to rename the duplicate file in some sort of sequence? That would be much appreciated :)) # Get all files Get-ChildItem "C:\zAa" -File -Recurse | ForEach-Object { # Get the modified date $dt = Get-Date $_.LastWriteTime $year = $dt.Year $month = $dt.Month # This adds "0" in front of the 1-9 months if($dt.Month -lt 10) { $month = "0" + $dt.Month.ToString() } else { $month = $dt.Month } # Remove leading '.' from the extension $extension = $_.Extension.Replace(".", "") # Where we want to move the file $destinationFolder = "C:\zBb\$extension\$year\$month\" # Ensure full folder path exists if(!(Test-Path $destinationFolder)) { New-Item -ItemType Directory -Force -Path $destinationFolder } # Copy/Move the item to it's new home Move-Item $_.FullName $destinationFolder } I haven't been able to do much, I normally go find the duplicates and rename them manually.
Probably the easiest way to move a file with a unique name is to use a Hashtable that stores the filenames already present. Then a simple loop can add a sequence number to its file name until it is no longer found in the Hashtable. Next simply move the file under that new name. Your code modified: # Where we want to move the file $destinationFolder = 'C:\zBb\{0}\{1:yyyy}\{1:MM}' -f $_.Extension.TrimStart("."), $_.LastWriteTime # Ensure full folder path exists $null = New-Item -Path $destinationFolder -ItemType Directory -Force # create a Hashtable and store the filenames already present in the destination folder $existing = #{} Get-ChildItem -Path $destinationFolder -File | ForEach-Object { $existing[$_.Name] = $true } # Get all source files Get-ChildItem "C:\zAa" -File -Recurse | ForEach-Object { # Copy/Move the item to it's new home # construct the new filename by appending an index number in between brackets $newName = $_.Name $count = 1 while ($existing.ContainsKey($newName)) { $newName = "{0}({1}){2}" -f $_.BaseName, $count++, $_.Extension } # add this new name to the Hashtable so it exists in the next run $existing[$newName] = $true # use Join-Path to create a FullName for the file $newFile = Join-Path -Path $destinationFolder -ChildPath $newName Write-Verbose "Moving '$($_.FullName)' as '$newFile'" $_ | Move-Item -Destination $newFile -Force }
Powershell script works locally but not under solarwinds SAM component
Below is the script which works locally under powershell ISE. But same under SAM Component ie Windows powershell monitor not working Script checks for doc file under path1 and if exist. It takes copy of the file from path1 and move to Path2 by renaming with timestamp. $directory = 'path1' $filter = '*.doc' $msg = ' ' $com = "," $count = 0 $stamp = Get-Date -F yyyy-MM-dd_HH-mm Get-ChildItem -Path $directory -Filter $filter | Foreach-Object { $msg = $msg + $com + $_.FullName $dest = "path1" + $_.Name $newName = "path1" + "$($_.BaseName)_$stamp$($_.Extension)" Copy-Item -Path $_.FullName -Destination $newName $count = $count + 1 } if($count -gt 0) { write-host 'Statistic.FileMonitor:' $count; write-host "Message.FileMonitor: $count file found and moved, Filenames- $msg" exit 0 } else { write-host 'Statistic.FileMonitor:' $count; write-host "Message.FileMonitor: $count files found, Filenames- $msg" exit 0 } When execution mode is set as Local Host. It execute. But it says no file found. Eventhough file exist under path1 When execution mode is seet as Remote. I am getting result as Down
Split csv file into specified number of files without page break
I have a 200,000 file to be split into 8 chunks using powershell The file has rows with the first value being the record 'KEY' I would like to ensure that rows corresponding to the key field value (which is the first value of the row) do not break across files when the split happens. Here is the simple split I use $i=0 Get-Content -Encoding Default "C:\Test.csv" -ReadCount 10130 | ForEach-Object { $i++ $_ | Out-File -Encoding Default "C:\Test_$i.csv" } Sample Data 0190709,HP16,B,B,3,3, 0190709,HP17,B,B,3,3, 0190709,HP18,B,B,3,3, 0196597,HP11,,CNN,,, 0196597,HP119,,CNN,,, 0196597,HP13,,CNN,,, 01919769,HP11,,ANN,,, 01919769,HP119,,OPN,,, 01919769,HP13,,CNN,,, 01919769,HP14,X,X,X,X, 01919769,HP15,A,A,X,X, 01919769,HP16,S,S,X,X, 01919769,HP17,S,S,5,5, 01919769,HP18,S,S,5,5, 0797819,HP14,X,AX,X,X, 0797819,HP15,X,XA,X,X, 0797819,HP16,X,X,XA,XA, 0797819,HP17,A,A,X,X, 0797819,HP18,A,A,AX,X, Expected Output Lets say we want 2 chunks of equal size. I would like 2 files like below with the key not split between files. Its ok if the file gets bigger (more lines) in an attempt to prevent page break of the key. File 1 0190709,HP16,B,B,3,3, 0190709,HP17,B,B,3,3, 0190709,HP18,B,B,3,3, 0196597,HP11,,CaweNN,,, 0196597,HP119,,CNN,,, 0196597,HP13,,CNwN,,, 01919769,HP11,,AawNN,,, 01919769,HP119,,OePN,,, 01919769,HP13,,CNN,,, 01919769,HP14,XY,X,X,X, 01919769,HP15,A,A,XC,XA, 01919769,HP16,S,S,X,X, 01919769,HP17,S,S,5A,5, 01919769,HP18,S,S,5,5, File 2 0797819,HP14,X,AX,X,X, 0797819,HP15,X,XA,X,X, 0797819,HP16,X,X,XA,XA, 0797819,HP17,A,A,X,X, 0797819,HP18,A,A,AX,X,
Although you have not supplied an example (first couple of lines) of your CSV file, the below function assumes the input csv file is valid. function Split-Csv { [CmdletBinding()] Param ( [Parameter(Mandatory = $true, Position = 0)] [string]$Path, # the full path and filename of the source CSV file [Parameter(Mandatory = $true, Position = 1)] [string]$Destination, # the path of the output folder [ValidateRange(1,[int]::MaxValue)] [int]$Chunks = 8, # the number of parts to split into [switch]$FirstLineHasHeaders ) # create the destination folder if it does not already exist if (!(Test-Path -Path $Destination -PathType Container)) { Write-Verbose "Creating folder '$Destination'" New-Item -Path $Destination -ItemType Directory | Out-Null } $outputFile = [System.IO.Path]::GetFileNameWithoutExtension($Path) $content = Get-Content -Path $Path $totalLines = $content.Count if ($FirstLineHasHeaders) { $headers = $content[0] $partsize = [Math]::Ceiling(($totalLines - 1) / $Chunks) for ($i = 0; $i -lt $Chunks; $i++) { $first = ($i * $partsize + 1) $last = [Math]::Min($first + $partsize -1, $totalLines - 1) $newFile = Join-Path -Path $Destination -ChildPath ('{0}-{1:000}.csv' -f $outputFile, ($i + 1)) Write-Verbose "Creating file '$newFile'" Set-Content -Path $newFile -Value $headers -Force Add-Content -Path $newFile -Value $content[$first..$last] } } else { $partsize = [Math]::Ceiling($totalLines / $Chunks) for ($i = 1; $i -le $Chunks; $i++) { $first = (($i - 1) * $partsize) $last = [Math]::Min(($i * $partsize) - 1, $totalLines - 1) $newFile = Join-Path -Path $Destination -ChildPath ('{0}-{1:000}.csv' -f $outputFile, $i) Write-Verbose "Creating file '$newFile'" Set-Content -Path $newFile -Value $content[$first..$last] -Force } } } If your input csv file has headers, you need to ensure every 'chunk' file also has these headers. Use the function WITH switch $FirstLineHasHeaders Split-Csv -Path 'C:\Test.csv' -Destination 'D:\test' -Chunks 8 -FirstLineHasHeaders -Verbose If your input csv file does NOT have headers, use it like: Split-Csv -Path 'C:\Test.csv' -Destination 'D:\test' -Chunks 8 -Verbose
How to create a copy using powershell without overwriting the original file
So lets say I want to copy the file test.txt to another folder, but I want it to put create a copy and not just erase the file. I know that Copy-Item overwrites the file in the destination folder but I don't want it to do that. It also has to be a function
I think this will help you: $destinationFolder = 'PATH OF THE DESTINATION FOLDER' $sourceFile = 'FULL PATH AND FILENAME OF THE SOURCE FILE' # split the filename into a basename and an extension variable $baseName = [System.IO.Path]::GetFileNameWithoutExtension($sourceFile) $extension = [System.IO.Path]::GetExtension($sourceFile) # you could also do it like this: # $fileInfo = Get-Item -Path $sourceFile # $baseName = $fileInfo.BaseName # $extension = $fileInfo.Extension # get an array of all filenames (name only) of the files with a similar name already present in the destination folder $allFiles = #(Get-ChildItem $destinationFolder -File -Filter "$baseName*$extension" | Select-Object -ExpandProperty Name) # construct the new filename $newName = $baseName + $extension $count = 1 while ($allFiles -contains $newName) { # add a sequence number in brackets to the end of the basename until it is unique in the destination folder $newName = "{0}({1}){2}" -f $baseName, $count++, $extension } # construct the new full path and filename for the destination of your Copy-Item command $targetFile = Join-Path -Path $destinationFolder -ChildPath $newName Copy-Item -Path $sourceFile -Destination $targetFile
Even if i isn't the most elegant way, you could try something like this $src = "$PSScriptRoot" $file = "test.txt" $dest = "$PSScriptRoot\dest" $MAX_TRIES = 5 $copied = $false for ($i = 1; $i -le $MAX_TRIES; $i++) { if (!$copied) { $safename = $file -replace "`.txt", "($i).txt" if (!(Test-Path "$dest\$file")) { Copy-Item "$src\$file" "$dest\$file" $copied = $true } elseif (!(Test-Path "$dest\$safename")) { Copy-Item "$src\$file" "$dest\$safename" $copied = $true } else { Write-Host "Found existing file -> checking for $safename" } } else { break } } The for-loop will try to safely copy the file up to 5 times (determined by $MAX_TRIES) If 5 times isn't enough, nothing will happen The regEx will create test(1).txt, test(2).txt, ... to check for a "safe" filename to copy The if-statement will check if the original file can be copied The elseif-statement will try to copy with the above created $safename The else-statement is just printing a "hint" on what's going on
Is there any easiest way to unzip folders and sub folders using powershell
hi i am trying to unzip folders and sub folders i have tried different ways and script but no luck. Can some one help on this $Date = Get-Date # $folder_date = $Date.ToString("yyyy-MM-dd_HHmmss") $content = get-childitem 'C:\Deployments' $sortedContent = $content | Sort-Object LastWriteTime -Descending # 1. LIST BACKUPS write-host "This is the list of all the Deployments :" $count = 1 foreach ($item in $sortedContent) { #Edit: Now with auto-incrementing counter Write-Host ("{0}: {1}" -f $count, $item.Name) $count++ } $itemNumber = Read-Host "Please select which deployments you want to copy " $itemNumber = $itemNumber -1 if($sortedContent[$itemNumber].PSIsContainer -eq $true) { $src = $sortedContent[$itemNumber].FullName + "\" $Date = Get-Date $folder_date = $Date.ToString("yyyy-MM-dd_HHmm") $tempPath = 'C:\_Temp\Restore' + $folder_date if (!(Test-Path -path $tempPath)) { New-Item $tempPath -type directory } $TabletzipPath = $src + "table.zip" $AdminzipPath = $src + "admin.zip" .\unzip.ps1 $src $tempPath .\unzip.ps1 $TabletzipPath $tempPath .\unzip.ps1 $AdminzipPath $tempPath }
$Shell = new-object -com shell.application $Zip = $Shell.NameSpace($PathToZip) ForEach($Item in $Zip.items()) { $Shell.Namespace($ExtractPath).copyhere($Item) }
If your just looking for an easy & quick way of doing this. You can use this function: Unzip-File -File C:\location.zip -Destination E:\destination Need this script: http://gallery.technet.microsoft.com/scriptcenter/PowerShell-Function-to-727d6200
Using jar unzipping: copy-paste it into PowerShell ISE or create a .ps1 file with this code: #choose directory where your script is cd $PSScriptRoot #do for each file in your directory, -recurse stands for all subfolders, *.jar stands for all files with jar extension foreach ($file in (dir -Recurse *.jar)) { #Uncomment this to check files this script is going to unzip (don't forget to comment 7z line below to avoid operation) #Write-Host ("Dir for file " + $file.Name + " is " + $file.Directory) #put here full path to 7z if there is no system env (or just make it) for that #x stands for eXtract files with full paths #-o is for output folder (example of usage -oc:\test\) 7z x $file.FullName ("-o"+$file.Directory) } It should unzip everything you need directly to the folder where your .jar (or any other archive) is.