We need to move .csv files from folder where the are stored down to external server using powershell.
this is what i've tried so far but for some reason i only get message not copying and name of the files:
$DestinationFolder = "C:\d1\"
$SourceFolder = "C:\s1\"
If (-not (Test-Path $DestinationFolder) ) {
New-Item -ItemType Directory -Force -Path $DestinationFolder
}
$EarliestModifiedTime = (Get-Date).AddMinutes(200).Date # get's current time + adds 30 min
$LatestModifiedTime = (Get-Date).Date
echo($DestinationFolder); # will check time
Get-ChildItem "C:\s1\*.*" |
ForEach-Object {
if ( ($_.CreationTime -ge $EarliestModifiedTime) -and ($_.CreationTime -lt $LatestModifiedTime) ){ # i.e., all day yesterday
Copy-Item $_ -Destination $DestinationFolder -Force
Write-Host "Copied $_" }
else {
Write-Host "Not copying $_"
}
}
does it work if you simplify it and just try for a test (e.g. pick one file rather than trying to run for a delta of every 30 mins / last day)? Just thinking first you need to see if the problem is with accessing (or the formatting) of your source / destination directories or the delta logic itself. Perhaps some more error conditions would help....
Related
Looking for suggestions on how to solve a little issue, I have powershell script I am working on, it is going to be run say 5 times a day, and will create a folder that is todays days and then like a .1 at the end of it, I will then copy files to that 1 specific folder, then the next time the script runs the folder will be .2 then copy files to that folder, but the problem that I am running into doing as if/elseif statements is the 2nd time it runs it says the .1 file already exists and stops running.
any thoughts on how to get around this?
here is my current set up
if ($Folder1 -eq ".\$((Get-Date).ToString('yyyy-MM-dd')).1") {
New-Item -ItemType Directory -Path ".\$((Get-Date).ToString('yyyy-MM-dd')).1"
$Folder1 = ".\$((Get-Date).ToString('yyyy-MM-dd')).1"
}
elseif ($Folder1 = ".\$((Get-Date).ToString('yyyy-MM-dd')).1"){
New-Item -ItemType Directory -Path ".\$((Get-Date).ToString('yyyy-MM-dd')).2"
$Folder2 = ".\$((Get-Date).ToString('yyyy-MM-dd')).2"
}
elseif ($Folder2 -eq ".\$((Get-Date).ToString('yyyy-MM-dd')).2"){
New-Item -ItemType Directory -Path ".\$((Get-Date).ToString('yyyy-MM-dd')).3"
$Folder3 = ".\$((Get-Date).ToString('yyyy-MM-dd')).3"
}
elseif ($Folder3 -eq ".\$((Get-Date).ToString('yyyy-MM-dd')).3"){
New-Item -ItemType Directory -Path ".\$((Get-Date).ToString('yyyy-MM-dd')).4"
$Folder4 = ".\$((Get-Date).ToString('yyyy-MM-dd')).4"
}
elseif ($Folder4 -eq ".\$((Get-Date).ToString('yyyy-MM-dd')).4"){
New-Item -ItemType Directory -Path ".\$((Get-Date).ToString('yyyy-MM-dd')).5"
$Folder5 = ".\$((Get-Date).ToString('yyyy-MM-dd')).5"
}
else {
"Path doesn't exist."
}
Thanks all!
maybe this might be what you need if I was able to understand your needs. I do validate if the folder for the current date with appended ".n" exists. The "n" is the current folder version starting with 1.
$folder = "c:\temp\"
$today = (Get-Date).ToString('yyyy-MM-dd')
do {
$i++
$NewFolder = Join-Path $folder ($today+".$i")
} while (Test-Path $NewFolder)
New-Item -Type Directory -Path $NewFolder | Out-Null
An-dir's answer is great, and I'm just going to offer a slightly more compact version because some people like that kind of thing.
$NewFolder=$Folder="C:\Temp\"+(Get-Date).ToString('yyyy-MM-dd')
While(Test-Path ($NewFolder="$Folder."+ ++$i)){}
$NewFolder = New-Item -Type Directory -Path $NewFolder
Like I said, same thing just a little more compact. Also shows a couple things I wasn't aware of for a long time with PowerShell, so I thought others might learn things like:
$This = $That = "Some Value"
Setting two variables in the same line to the same thing.
While(Test-Path ($NewFolder="$Folder."+ ++$i))
Assigning a value to a variable, and being able to test against that value at the same time. Works with If, Where, and all those kinds of things that allow you to perform an evaluation that I've seen.
I want to create a 0-filesize mirror image of a set of folder, but while robocopy is really good, it doesn't save all of the information that I would like:
robocopy D:\documents E:\backups\documents_$(Get-Date -format "yyyyMMdd_HHmm")\ /mir /create
The /create switch makes each file in the duplicate folder have zero-size, and that is good, but I would like each file in the duplicate folder to have [size] appended to the end of the name with the size in KB or MB or GB, and the create / last modified time on every file to exactly match the original file. This way, I will have a zero-size duplicate of the folder that I can archive, but which contains all of the relevant information for the files in that directory, showing the size of each and the exact create / last modified times.
Are there good / simple ways to iterate through a tree in PowerShell, and for each item create a zero size file with all relevant information like this?
This would be one way to implement the copy command using the approach I mentioned in the comments. This should give you something to pull ideas from. I didn't intend to spend as much time on it as I did, but I ran it on several directories and found some problems and debugged each problem I encountered. This is a pretty solid example at this point.
function Copy-FolderZeroSizeFiles {
[CmdletBinding()]
param( [Parameter(Mandatory)] [string] $FolderPath,
[Parameter(Mandatory)] [string] $DestinationPath )
$dest = New-Item $DestinationPath -Type Directory -Force
Push-Location -LiteralPath $FolderPath
try {
foreach ($item in Get-ChildItem '.' -Recurse) {
$relPath = Resolve-Path -LiteralPath $item -Relative
$type = if ($item.Attributes -match 'Directory')
{ 'Directory' }
else { 'File' }
$destItem = New-Item "$dest\$relPath" -Type $type -Force
$destItem.Attributes = $item.Attributes
$destItem.LastWriteTime = $item.LastWriteTime
}
} finally {
Pop-Location
}
}
Note: the above implementation is simplistic and represents anything that isn't a directory as a file. That means symbolic links, et al. will be files with no information what they would be linked to.
Here's a function to get the conversion from number of bytes to N.N B/K/M/G format. To get more decimal places, just add 0's to the end of the format strings.
function ConvertTo-FriendlySize($NumBytes) {
switch ($NumBytes) {
{$_ -lt 1024} { "{0,7:0.0}B" -f ($NumBytes) ; break }
{$_ -lt 1048576} { "{0,7:0.0}K" -f ($NumBytes / 1024) ; break }
{$_ -lt 1073741824} { "{0,7:0.0}M" -f ($NumBytes / 1048576) ; break }
default { "{0,7:0.0}G" -f ($NumBytes / 1073741824); break }
}
}
Often, people get these conversions wrong. For instance, it's a common error to use 1024 * 1000 to get Megabytes (which is mixing the base10 value for 1K with the base2 value for 1K) and follow that same logic to get GB and TB.
Here is what I came up with with the additional parts in the question, change $src / $dst as required (D:\VMs is where I keep a lot of Virtual Machines). I have included setting all of CreationTime, LastWriteTime, LastAccessTime so that the backup location with zero-size files is a perfect representation of the source. As I want to use this for archival purposes, I have finally zipped things up and included a date-time stamp in the zipfile name.
# Copy-FolderZeroSizeFiles
$src = "D:\VMs"
$dst = "D:\VMs-Backup"
function ConvertTo-FriendlySize($NumBytes) {
switch ($NumBytes) {
{$_ -lt 1024} { "{0:0.0}B" -f ($NumBytes) ; break } # Change {0: to {0,7: to align to 7 characters
{$_ -lt 1048576} { "{0:0.0}K" -f ($NumBytes / 1024) ; break }
{$_ -lt 1073741824} { "{0:0.0}M" -f ($NumBytes / 1048576) ; break }
default { "{0:0.0}G" -f ($NumBytes / 1073741824); break }
}
}
function Copy-FolderZeroSizeFiles($FolderPath, $DestinationPath) {
Push-Location $FolderPath
if (!(Test-Path $DestinationPath)) { New-Item $DestinationPath -Type Directory }
foreach ($item in Get-ChildItem $FolderPath -Recurse -Force) {
$relPath = Resolve-Path $item.FullName -Relative
if ($item.Attributes -match 'Directory') {
$new = New-Item "$DestinationPath\$relPath" -ItemType Directory -Force -EA Silent
}
else {
$fileBaseName = [System.IO.Path]::GetFileNameWithoutExtension($item.Name)
$fileExt = [System.IO.Path]::GetExtension($item.Name)
$fileSize = ConvertTo-FriendlySize($item.Length)
$new = New-Item "$DestinationPath\$(Split-Path $relPath)\$fileBaseName ($fileSize)$fileExt" -ItemType File
}
"$($new.Name) : creation $($item.CreationTime), lastwrite $($item.CreationTime), lastaccess $($item.LastAccessTime)"
$new.CreationTime = $item.CreationTime
$new.LastWriteTime = $item.LastWriteTime
$new.LastAccessTime = $item.LastAccessTime
$new.Attributes = $item.Attributes # Must this after setting creation/write/access times or get error on Read-Only files
}
Pop-Location
}
Copy-FolderZeroSizeFiles $src $dst
$dateTime = Get-Date -Format "yyyyMMdd_HHmm"
$zipName = "$([System.IO.Path]::GetPathRoot($dst))\$([System.IO.Path]::GetFileName($dst))_$dateTime.zip"
Add-Type -AssemblyName System.IO.Compression.FileSystem
[IO.Compression.ZipFile]::CreateFromDirectory($dst, $zipName)
I am needing to write a script to automate moving all of the IIS log files from a system that has been running for months. This system consists of many web servers.
I am successful at getting the log directory moved to the new location, and then my script will move the other log files from the old directory to the new location with the exception of the current file. It cannot move because there is a file in the destination with the same name. (current log file.) I want to rename it, but do not want to rename all files in the directory. I know I could use a wildcard, but would prefer to rename only that file.
What command can I use to find the name of the file, not the directory or path? I have pieced this together from other smaller requests I have found on here and on the web.
Import-Module WebAdministration
$LogPath = "e:\Logs\IISlogs"
foreach($WebSite in $(get-website))
{
$logFile="$($Website.logFile.directory)\w3svc$($website.id)".replace("%SystemDrive%",$env:SystemDrive)
}
New-Item $LogPath -ItemType Directory
Set-ItemProperty “IIS:\Sites\Default Web Site” -name logFile.directory -value $LogPath
$path = $logpath+"\W3SVC1"
$Timeout = 60
$timer = [Diagnostics.Stopwatch]::StartNew()
while (($timer.Elapsed.TotalSeconds -lt $Timeout) -and (-not (Test-Path -Path $path )))
{
Start-Sleep -Seconds 1
$tot = $timer.Elapsed.Seconds
Write-Output -Message ("Still waiting for action to complete after " + $tot + " seconds upto " + $Timeout)
}
$timer.Stop()
Move "$($logfile)\*.*" $path
All files have a time a date stamp. Filter the file name, sort by the time stamp descending, and select the first one.
For example:
$dir = "e:\Logs\IISlogs"
($latest = Get-ChildItem -Path $dir |
Sort-Object LastAccessTime -Descending |
Select-Object -First 1)
"Current log file details: $($latest.name)"
Get-Content $latest
If you are saying the logs can be of a different name, then you need to specify the wildcard match for that in the path specification.
(I have tried to rewrite this to be more clear)
I have a problem with having hundreds of folders and files (each start with ABC and then numbers 001-936) that need to be moved to a single folder with the same name. For example:
C:\folders\ABC001
C:\folders\random\ABC001\
C:\folders\files\ABC001.pdf
C:\folders\ABC002.pdf
C:\folders\ABC002\
C:\folders\needs sorting\ABC002\
That I would like to move all files and folders to a folder with the same name:
C:\folders\ABC\ABC001\
C:\folders\ABC\ABC002\
and so on...
However, sometimes the orginal folder and/or file will have a duplicate in the destination folder(or may already have since folders already exist), so I want it to add to the name (1), (2), etc.
C:\folders\ABC\ABC001\ABC001\
C:\folders\ABC\ABC001\ABC001 (1)\
C:\folders\ABC\ABC001\ABC001.pdf
C:\folders\ABC\ABC002\ABC002.pdf
C:\folders\ABC\ABC002\ABC002\
C:\folders\ABC\ABC002\ABC002 (1)\
Any help would be greatly appreciated, I have been trying to solve this for weeks (everytime I needed the files) but am new to scripting/code. I started with:
for /R "C:\folders" %x in (*abc123*.*) do move "%x" "D:\folders\abc\abc123"
Closest attempt (minor edits to another code):
function MoveFileToFolder ([string]$source,[string]$destination){
# get a list of source files
$files = Get-ChildItem -Path $source -Recurse | ?{($_.PSIsContainer)}
# verify if the list of source files is empty
if ($files -ne $null) {
foreach ($file in $files) {
$filename = $file.Name
$filebase = $file.BaseName
$fileext = $file.Extension
# verify if destination file exists
$filenameNU = $filename
if (Test-Path (Join-Path $destination $filename)) {
$n = 0
while ((Test-Path (Join-Path $destination $filenameNU)) -eq $true)
{
$filenameNU = $filebase + "(" + ++$n + ")" + $fileext
}
}
Move-Item $file.FullName (Join-Path $destination $filenameNU)
}
}
}
MoveFileToFolder C:\folders\ C:\folders\abc\abc123
(I would try and run this in Powershell for each subfolder, but it became very hard to keep up with)
Thank you for reading.
I have a folder that contains many huge files. I want to split these files in 3 folders. The requirement is to get the count of files in main folder and then equally split those files in 3 child folders.
Example - Main folder has 100 files. When I run the powershell, 3 child folders should be created with 33, 33 and 34 files respectively.
How can we do this using Powershell?
I've tried the following:
$FileCount = (Get-ChildItem C:\MainFolder).count
Get-ChildItem C:\MainFolder -r | Foreach -Begin {$i = $j = 0} -Process {
if ($i++ % $FileCount -eq 0) {
$dest = "C:\Child$j"
md $dest
$j++
}
Move-Item $_ $dest
}
Here is another solution. This one accounts for the sub folders not existing.
# Number of groups to support
$groupCount = 3
$path = "D:\temp\testing"
$files = Get-ChildItem $path -File
For($fileIndex = 0; $fileIndex -lt $files.Count; $fileIndex++){
$targetIndex = $fileIndex % $groupCount
$targetPath = Join-Path $path $targetIndex
If(!(Test-Path $targetPath -PathType Container)){[void](new-item -Path $path -name $targetIndex -Type Directory)}
$files[$fileIndex] | Move-Item -Destination $targetPath -Force
}
If you need to split up the files into a different number of groups the use $groupCount of higher that 3. Can also work logic with a switch that would change $groupCount to something else if the count was greater that 500 for example.
Loop though the files one by one. Using $fileIndex as a tracker we determine the folder 0,1 or 2 in my case that the file will be put into. Then using that value we check to be sure the target folder exists. Yes, this logic could easily be placed outside the loop but if you have file and folder changes while the script is running you could argue it is more resilient.
Ensure the folder exists, if not make it. Then move that one item. Using the modulus operator, like in the other answers, we don't have to worry about how many files are there. Let PowerShell do the math.
This is super quick and dirty, but it does the job.
#Get the collection of files
$files = get-childitem "c:\MainFolder"
#initialize a counter to 0 or 1 depending on if there is a
#remainder after dividing the number of files by 3.
if($files.count % 3 -eq 0){
$counter = 0
} else {
$counter = 1
}
#Iterate through the files
Foreach($file in $files){
#Determine which subdirectory to put the file in
If($counter -lt $files.count / 3){
$d = "Dir1"
} elseif($counter -ge $files.count / 3 * 2){
$d = "Dir3"
} else {
$d = "Dir2"
}
#Create the subdirectory if it doesn't exist
#You could just create the three subdirectories
#before the loop starts and skip this
if(-Not (test-path c:\Child\$d)){
md c:\Child\$d
}
#Move the file and increment the counter
move-item $file.FullName -Destination c:\Child\$d
$counter ++
}
I think it's possible to do without doing the counting and allocating yourself. This solution:
Lists all the files
Adds a counter property which cycles 0,1,2,0,1,2,0,1,2 to each file
groups them into buckets based on the counter
moves each bucket in one command
There's scope for rewriting it in a lot of ways to make it nicer, but this saves doing the math, handling uneven allocations, iterating over the files and moving them one at a time, would easily adjust to different numbers of groups.
$files = (gci -recurse).FullName
$buckets = $files |% {$_ | Add-Member NoteProperty "B" ($i++ % 3) -PassThru} |group B
$buckets.Name |% {
md "c:\temp$_"
Move-Item $buckets[$_].Group "c:\temp$_"
}