I have a script that needs to copy a list of files to ceraitn directories and locations on target servers.
I was able to understand that I need to creat an csv file as follows:
I need to understand from you how to make the files from source location to their adjacent file in target location. Any ideas?
My code looks like this:
# customize log file
$date = (Get-Date -Format d).ToString() | foreach {$_ -replace "/", "_"}
$time = (Get-Date)
$scriptDir = "D:\Scripts\ServerBuildToolkitT1\SingleFileUpfate\"
$logDir = "D:\Scripts\ServerBuildToolkitT1\Logs\SingleFileUpfate\"
$logFileName = "SingleFileUpfate $date.log"
$sources = #()
$destinsyions = #()
function CSV {
Import-Csv D:\Scripts_PS\SD.csv | ForEach-Object {
$sources += $_."Source Location"
$destinations += $_."Destination Location"
}
}
# this file contains the list of destination server that you want copy
# file/folder to
$computers = Get-Content "D:\Scripts_PS\ServerList.txt"
function main
{
foreach ($computer in $computers) {
foreach ($destination in $destinations) {
Write-Output "$([DateTime]::Now) Copying files update to $computer now" |
Out-File -FilePath "$logDir\$logFileName" -Append
Copy-Item $sources -Destination "\\$computer\$destination" -Force -Recurse -Verbose:$false
}
}
}
csv
main
Write-Output "$([DateTime]::Now) the operation SingleFileUpdate completed successefully" |
Out-File -FilePath "$logDir\$logFileName" -Append
i have updtaed the Script (as seen above) and now i am getting the following ERROR
"WARNING: One or more headers were not specified. Default names starting with "H" have been used in place of any missing headers."
Related
I am adding an email function to send an email if the file uploads success or failure, but when I upload the file, it sends a message as I write in the body for success but I am not sure if it captures the error function as well in the email.
I upload the file as per the naming convention and it moved successfully.
When I tried to upload the wrong file through this, it does not capture the catch statement in the log file.
I want to capture the backup copy name in logs as well.
Got an email on the successful event but didn't get an email on the catch statement.
Please see the below code.
$Source = 'c:\uploadtool\' # Source Location
$RetailSource = 'Retail P&C Sales Intelligence\*'
$GroupSource = 'Group P&C Sales Intelligence\*'
$RetailDest = 'D:\ToolUpload\Retail-EIP' # 1st Destination Location
$GroupDest = 'D:\ToolUpload\Group-EIP' # 2nd Destination location
$ArchiveData = 'D:\Backup\backup_{0:yyyyMMddHHmm}' -f (Get-Date)
$LogFolder = 'D:\logs'
# because of your wish to get a Table-style log, use CSV format you can open in Excel
$LogFile = 'D:\logs\uploadlog_{0:yyyyMMdd}.csv'-f (Get-Date)
$Sourcetest = Test-Path -Path '$RetailSource','$GroupSource' -PathType Leaf
$SmtpServer ='essexlake1.mail.protection.outlook.com' # SMTP Server name
$SmtpPort='25'
$Subject='File Upload Status'
$To="####domain.com"
$From="#####domain.com"
# make sure the output LogFolder exist
# by adding the -Force switch there is no need to use Test-Path first, because if
# the folder already exists, the cmdlet will return the DirectoryInfo of that,
# otherwise it will create a new folder. Since we dont want output, we use $null = ..
$null = New-Item -Path $LogFolder -ItemType Directory -Force
# loop through the files in the source folder and collect the outputted objects
$result = Get-ChildItem -Path $Sourcetest -Include '*Group-EIP*', '*Retail-EIP*' -File -Force -Recurse |
ForEach-Object {
Write-Host "Processing file '$($_.FullName)'"
# create an object with (for now) 3 empty properties
$out = $_ | Select-Object #{Name = 'Date'; Expression = {(Get-Date)}},
#{Name = 'Source'; Expression = {$_.FullName}},
#{Name = 'FileSize'; Expression = {$_.Length}},
Destination, # depends on the file name
#{Name = 'Archive'; Expression = {$ArchiveData}}, # initialize to Not Applicable
Result
# depending on its name, get the correct destination folder
$destFolder = if($_.Name -match "Retail-EIP") { $RetailDest } else { $GroupDest }
# create the backup destination folder if it didn't already exist
# the first file in column 'Source' is now responsible for creating the backup folder
$null = New-Item -Path $destFolder -ItemType Directory -Force
# get the full path and filename for the destination
$existingFile = Join-Path -Path $destFolder -ChildPath $_.Name
# add the destination folder to the output object
$out.Destination = $destFolder
try {
# if a file with that name already exists in the destination, move it to the Archive folder
if (Test-Path -Path $existingFile -PathType Leaf) {
# create the Archive folder if it didn't already exist
$null = New-Item -Path $ArchiveData -ItemType Directory -Force
Move-Item -Path $existingFile -Destination $ArchiveData -ErrorAction Stop
# add the archived file to the output object
$out.Archive = $existingFile
Write-Host "File '$existingFile' has been backed-up to '$ArchiveData'"
}
# next move the file from the source folder to its destination (either $RetailDest or $GroupDest)
$_ | Move-Item -Destination $destFolder -ErrorAction Stop
$out.Result = 'OK'
Write-Host "File '$($_.FullName)' has been moved to '$destFolder'"
$Body=" The File '$($_.FullName)' has been moved to '$destFolder"
Send-MailMessage -SmtpServer $SmtpServer -Port $Port -From $From -To $To -Subject $Subject -Body $Body
}
catch {
# ouch.. something went horribly wrong on a Move-Item action
Write-Warning "An error occurred: $_.Exception.Message"
$out.Result = "Error: $($_.Exception.Message)" | Add-Content -Path $LogFile -Force
}
# output the object so it gets collected in variable $result
$out
}
# now you can save the results as structured CSV file to open in Excel
$result | Export-Csv -Path $LogFile -UseCulture -NoTypeInformation -Append
# and display on screen using Out-GridView as the data will probably be too wide for Format-Table
$result | Out-GridView -Title 'Backup results
'
It's not going to send an email on-error because Send-MailMessage is the last statement in the Try{...} block. If anything errors before that the statement Send-MailMessage won't run.
If it were me, and from this starting point I'd populate $Body respective to the outcome (success/fail) and send the email afterward. Not only would this send with correct information regardless but avoids having a second Send-MailMessage command in the catch block. Obviously I can't rig up a test of such a big code segment, but it might look something like:
$Source = 'c:\uploadtool\' # Source Location
$RetailSource = 'Retail P&C Sales Intelligence\*'
$GroupSource = 'Group P&C Sales Intelligence\*'
$RetailDest = 'D:\ToolUpload\Retail-EIP' # 1st Destination Location
$GroupDest = 'D:\ToolUpload\Group-EIP' # 2nd Destination location
$ArchiveData = 'D:\Backup\backup_{0:yyyyMMddHHmm}' -f (Get-Date)
$LogFolder = 'D:\logs'
# because of your wish to get a Table-style log, use CSV format you can open in Excel
$LogFile = 'D:\logs\uploadlog_{0:yyyyMMdd}.csv'-f (Get-Date)
$Sourcetest = Test-Path -Path '$RetailSource','$GroupSource' -PathType Leaf
#Email Params:
$EmailParams = #{
SmtpServer ='essexlake1.mail.protection.outlook.com' # SMTP Server name
Port ='25'
Subject ='File Upload Status'
To ="####domain.com"
From ="#####domain.com"
}
# make sure the output LogFolder exist
# by adding the -Force switch there is no need to use Test-Path first, because if
# the folder already exists, the cmdlet will return the DirectoryInfo of that,
# otherwise it will create a new folder. Since we dont want output, we use $null = ..
$null = New-Item -Path $LogFolder -ItemType Directory -Force
# loop through the files in the source folder and collect the outputted objects
$result =
Get-ChildItem -Path $Sourcetest -Include '*Group-EIP*', '*Retail-EIP*' -File -Force -Recurse |
ForEach-Object {
Write-Host "Processing file '$($_.FullName)'"
# create an object with (for now) 3 empty properties
$out = $_ |
Select-Object #{Name = 'Date'; Expression = {(Get-Date)}},
#{Name = 'Source'; Expression = {$_.FullName}},
#{Name = 'FileSize'; Expression = {$_.Length}},
Destination, # depends on the file name
#{Name = 'Archive'; Expression = {$ArchiveData}}, # initialize to Not Applicable
Result
# depending on its name, get the correct destination folder
$destFolder = if($_.Name -match "Retail-EIP") { $RetailDest } else { $GroupDest }
# create the backup destination folder if it didn't already exist
# the first file in column 'Source' is now responsible for creating the backup folder
$null = New-Item -Path $destFolder -ItemType Directory -Force
# get the full path and filename for the destination
$existingFile = Join-Path -Path $destFolder -ChildPath $_.Name
# add the destination folder to the output object
$out.Destination = $destFolder
try
{ # if a file with that name already exists in the destination, move it to the Archive folder
if (Test-Path -Path $existingFile -PathType Leaf) {
# create the Archive folder if it didn't already exist
$null = New-Item -Path $ArchiveData -ItemType Directory -Force
Move-Item -Path $existingFile -Destination $ArchiveData -ErrorAction Stop
# add the archived file to the output object
$out.Archive = $existingFile
Write-Host "File '$existingFile' has been backed-up to '$ArchiveData'"
}
# next move the file from the source folder to its destination (either $RetailDest or $GroupDest)
$_ | Move-Item -Destination $destFolder -ErrorAction Stop
$out.Result = 'OK'
Write-Host "File '$($_.FullName)' has been moved to '$destFolder'"
$Body = " The File '$($_.FullName)' has been moved to '$destFolder"
}
catch
{ # ouch.. something went horribly wrong on a Move-Item action
$Body = " Error occured trying to move the file '$($_.FullName)' to '$destFolder. `n$($_.Exception.Message)"
Write-Warning "An error occurred: $_.Exception.Message"
$out.Result = "Error: $($_.Exception.Message)" | Add-Content -Path $LogFile -Force
}
Send-MailMessage #EmailParams -Body $Body
# output the object so it gets collected in variable $result
$out
}
# now you can save the results as structured CSV file to open in Excel
$result | Export-Csv -Path $LogFile -UseCulture -NoTypeInformation -Append
# and display on screen using Out-GridView as the data will probably be too wide for Format-Table
$result | Out-GridView -Title 'Backup results'
Forgive the reformat, that was for my own readability. I also added splatting to the email command. You were setting all those parameter vars early in the script any way...
Note: for all other parts of this question I think the straight forward answer is you have to add the instructions to tell the program what you want it to do. If you want to log on a specific condition, figure out where that's going to happen and add an Add-Content command etc...
I am trying to construct a script that moves through specific folders and the log files in it, and filters the error codes. After that it passes them into a new file.
I'm not really sure how to do that with for loops so I'll leave my code bellow.
If someone could tell me what I'm doing wrong, that would be greatly appreciated.
$file_name = Read-Host -Prompt 'Name of the new file: '
$path = 'C:\Users\user\Power\log_script\logs'
Add-Type -AssemblyName System.IO.Compression.FileSystem
function Unzip
{
param([string]$zipfile, [string]$outpath)
[System.IO.Compression.ZipFile]::ExtractToDirectory($zipfile, $outpath)
}
if ([System.IO.File]::Exists($path)) {
Remove-Item $path
Unzip 'C:\Users\user\Power\log_script\logs.zip' 'C:\Users\user\Power\log_script'
} else {
Unzip 'C:\Users\user\Power\log_script\logs.zip' 'C:\Users\user\Power\log_script'
}
$folder = Get-ChildItem -Path 'C:\Users\user\Power\log_script\logs\LogFiles'
$files = foreach($logfolder in $folder) {
$content = foreach($line in $files) {
if ($line -match '([ ][4-5][0-5][0-9][ ])') {
echo $line
}
}
}
$content | Out-File $file_name -Force -Encoding ascii
Inside the LogFiles folder are three more folders each containing log files.
Thanks
Expanding on a comment above about recursing the folder structure, and then actually retrieving the content of the files, you could try something line this:
$allFiles = Get-ChildItem -Path 'C:\Users\user\Power\log_script\logs\LogFiles' -Recurse
# iterate the files
$allFiles | ForEach-Object {
# iterate the content of each file, line by line
Get-Content $_ | ForEach-Object {
if ($_ -match '([ ][4-5][0-5][0-9][ ])') {
echo $_
}
}
}
It looks like your inner loop is of a collection ($files) that doesn't yet exist. You assign $files to the output of a ForEach(...) loop then try to nest another loop of $files inside it. Of course at this point $files isn't available to be looped.
Regardless, the issue is you are never reading the content of your log files. Even if you managed to loop through the output of Get-ChildItem, you need to look at each line to perform the match.
Obviously I cannot completely test this, but I see a few issues and have rewritten as below:
$file_name = Read-Host -Prompt 'Name of the new file'
$path = 'C:\Users\user\Power\log_script\logs'
$Pattern = '([ ][4-5][0-5][0-9][ ])'
if ( [System.IO.File]::Exists( $path ) ) { Remove-Item $path }
Expand-Archive 'C:\Users\user\Power\log_script\logs.zip' 'C:\Users\user\Power\log_script'
Select-String -Path 'C:\Users\user\Power\log_script\logs\LogFiles\*' -Pattern $Pattern |
Select-Object -ExpandProperty line |
Out-File $file_name -Force -Encoding ascii
Note: Select-String cannot recurse on its own.
I'm not sure you need to write your own UnZip function. PowerShell has the Expand-Archive cmdlet which can at least match the functionality thus far:
Expand-Archive -Path <SourceZipPath> -DestinationPath <DestinationFolder>
Note: The -Force parameter allows it to over write the destination files if they are already present. which may be a substitute for testing if the file exists and deleting if it does.
If you are going to test for the file that section of code can be simplified as:
if ( [System.IO.File]::Exists( $path ) ) { Remove-Item $path }
Unzip 'C:\Users\user\Power\log_script\logs.zip' 'C:\Users\user\Power\log_script'
This is because you were going to run the UnZip command regardless...
Note: You could also use Test-Path for this.
Also there are enumerable ways to get the matching lines, here are a couple of extra samples:
Get-ChildItem -Path 'C:\Users\user\Power\log_script\logs\LogFiles' |
ForEach-Object{
( Get-Content $_.FullName ) -match $Pattern
# Using match in this way will echo the lines that matched from each run of
# Get-Content. If nothing matched nothing will output on that iteration.
} |
Out-File $file_name -Force -Encoding ascii
This approach will read the entire file into an array before running the match on it. For large files it may pose a memory issue, however it enabled the clever use of -match.
OR:
Get-ChildItem -Path 'C:\Users\user\Power\log_script\logs\LogFiles' |
Get-Content |
ForEach-Object{ If( $_ -match $Pattern ) { $_ } } |
Out-File $file_name -Force -Encoding ascii
Note: You don't need the alias echo or its real cmdlet Write-Output
UPDATE: After fuzzing around a bit and trying different things I finally got it to work.
I'll include the code below just for demonstration purposes.
Thanks everyone
$start = Get-Date
"`n$start`n"
$file_name = Read-Host -Prompt 'Name of the new file: '
Out-File $file_name -Force -Encoding ascii
Expand-Archive -Path 'C:\Users\User\Power\log_script\logs.zip' -Force
$i = 1
$folders = Get-ChildItem -Path 'C:\Users\User\Power\log_script\logs\logs\LogFiles' -Name -Recurse -Include *.log
foreach($item in $folders) {
$files = 'C:\Users\User\Power\log_script\logs\logs\LogFiles\' + $item
foreach($file in $files){
$content = Get-Content $file
Write-Progress -Activity "Filtering..." -Status "File $i of $($folders.Count)" -PercentComplete (($i / $folders.Count) * 100)
$i++
$output = foreach($line in $content) {
if ($line -match '([ ][4-5][0-5][0-9][ ])') {
Add-Content -Path $file_name -Value $line
}
}
}
}
$end = Get-Date
$time = [int]($end - $start).TotalSeconds
Write-Output ("Runtime: " + $time + " Seconds" -join ' ')
I want to check .jpg file in the 2nd folder. 2nd folder has some subfolder. if .jpg exist in the subfolder of 2nd folder, I will copy a file from 1st folder to subfolder of 2nd folder based on the base name. I can do this part refer to this answer
How to copy file based on matching file name using PowerShell?
But I want to do limitation while I copy a file from 1st folder. I will NOT copy the file, if I already copy the same file from 1st folder to 2nd folder 3 times.
This is the code from the reference.
$Job_Path = "D:\Initial"
$JobError = "D:\Process"
Get-ChildItem -Path "$OpJob_Path\*\*.jpg" | ForEach-Object {
$basename = $_.BaseName.Substring(15)
$job = "$Job_Path\${basename}.png"
if (Test-Path $job) {
$timestamp = Get-Date -Format 'yyyyMMddhhmmss'
$dst = Join-Path $_.DirectoryName "${timestamp}_${basename}.gif"
Copy-Item $job $dst -Force
}
Anyone can help me to solve this problem please. Thank you.
Updated
$Job_Path = "D:\Initial"
$JobError = "D:\Process"
Get-ChildItem -Path "$OpJob_Path\*\*.jpg" | ForEach-Object {
$basename = $_.BaseName.Substring(15)
$job = "$Job_Path\${basename}.png"
if (Test-Path $job) {
$timestamp = Get-Date -Format 'yyyyMMddhhmmss'
$dst = Join-Path $_.DirectoryName "${timestamp}_${basename}.gif"
$Get = (Get-ChildItem -Name "$OpJob_Path\*\*$basename.jpg*" | Measure-Object).Count
$Get
if ($Get -eq "3") {
Write-Host "Continue Other Process"
NEXT_PROCESS
} else {
Write-Host "Less than 3"
}
Copy-Item $job $dst -Force
}
$Get is not a hashtable, and it's also not keeping track of what's been copied already. You need to define the hashtable outside the loop
$copy_count = #{}
Get-ChildItem -Path "$OpJob_Path\*\*.jpg" | ForEach-Object {
...
}
and then update it whenever you copy a file
if ($copy_count[$_.Name] -le 3) {
Copy-Item ...
$copy_count[$_.Name]++
} else {
...
}
I have many file in a folder, I would like to check the existing and matching of the file with variable that I initialize. Then, if the file exit and match, I want to get some information from the files (many file), then create a new file depend on how many file exist and match.
I tried this code, I can check the matching and existing file. I can create a new file and get the information from the file, but I only can create 1 file.
The information that I get from the file, each file is different.
$ID = "123"
$Pre = "ABC"
$Path = "C:\Folder"
$PO = Get-ChildItem -Path $Path
foreach ($File in $PO) {
if (($File.Name -match $ID) -and ($File.Name -match $Pre)) {
Write-Host ">>POfile Found: $File"
} else {
Write-Host ">>Check Again!"
}
}
# CREATE FILE
$Jb_Path = "C:\Folder\Jb"
## GET INFORMATION
$count = 1
$Get_PO = Get-ChildItem -Path $Path\$File -File -Recurse
$POfile = Get-Random -InputObject $Get_PO -Count $count
Write-Host ">>Selected POfile= $POfile"
$FilteredContents = Get-Content $POfile | Where-Object {$_ -like "*;INFO*"}
$Get_INFO = $FilteredContents.Substring(5,2)
## NEW FILE
New-Item -Path $Jb_Path\NEW_$Pre$ID-$Get_INFO.txt -Force
In the section # CREATE FILE you are referencing the variable $File which has the last value iterated in the previous foreach (even if it didn't match the if condition).
Asuming the $Pre is for prefix and comes first in a file name simply do a
Get-ChildItem "$Path\*$Pre*$ID*"
to only get file names for your criteria.
As $File contains only one file name a Get-Random doesn't make sense, especially as it might not contain a line with ;INFO
Assuming the two characters to extract are in front of ;INFO this untested script might do:
$Pre = "ABC"
$ID = "123"
$Path = "C:\Folder"
$Jb_Path= "C:\Folder\Jb"
Get-ChildItem "$Path\*$Pre*$ID*" | Get-Content |
Select-String -Pattern '^.....(..).*;INFO' |
Get-Random | ForEach-Object {
$NewFile = Join-Path $Jb_Path ('NEW_{0}{1}-{2}.txt' -f $Pre,
$ID,$_.Matches.Groups[1].Value)
New-Item -Path $NewFile -ItemType File -Force -WhatIf
}
It will only output what it would do without the -WhatIf parameter.
If no file matching the criteria and RegEx pattern is found it will silently continue.
If my assumptions led me wrong, enhance your question be editing it with more details.
Really need help creating a script that backs up, and shoots out the error along the file that did not copy
Here is what I tried:
Creating lists of filepaths to pass on to copy-item, in hopes to later catch errors per file, and later log them:
by using $list2X I would be able to cycle through each file, but copy-item loses the Directory structure and shoots it all out to a single folder.
So for now I am using $list2 and later I do copy-item -recurse to copy the folders:
#create list to copy
$list = Get-ChildItem -path $source | Select-Object Fullname
$list2 = $list -replace ("}"),("")
$list2 = $list2 -replace ("#{Fullname=") , ("")
out-file -FilePath g:\backuplog\DirList.txt -InputObject $list2
#create list crosscheck later
$listX = Get-ChildItem -path $source -recurse | Select-Object Fullname
$list2X = $listX -replace ("}"),("")
$list2X = $list2X -replace ("#{Fullname=") , ("")
out-file -FilePath g:\backuplog\FileDirList.txt -InputObject $list2X
And here I would pass the list:
$error.clear()
Foreach($item in $list2){
Copy-Item -Path $item -Destination $destination -recurse -force -erroraction Continue
}
out-file -FilePath g:\backuplog\errorsBackup.txt -InputObject $error
Any help with this is greatly appreciated!!!
The answer to complex file-copying or backup scripts is almost always: "Use robocopy."
Bill
"Want to copy all the items in C:\Scripts (including subfolders) to C:\Test? Then simply use a wildcard character..."
Next make it easier on yourself and do something like this:
$files = (Get-ChildItem $path).FullName #Requires PS 3.0
#or
$files = Get-ChildItem $path | % {$_.Fullname}
$files | Out-File $outpath
well it took me a long time, considering my response time. here is my copy function, which logs most errors(network drops, failed copies , etc) the copy function , and targetobject.
Function backUP{ Param ([string]$destination1 ,$list1)
$destination2 = $destination1
#extract new made string for backuplog
$index = $destination2.LastIndexOf("\")
$count = $destination2.length - $index
$source1 = $destination2.Substring($index, $count)
$finalstr2 = $logdrive + $source1
Foreach($item in $list1){
Copy-Item -Container: $true -Recurse -Force -Path $item -Destination $destination1 -erroraction Continue
if(-not $?)
{
write-output "ERROR de copiado : " $error| format-list | out-file -Append "$finalstr2\GCI-ERRORS-backup.txt"
Foreach($erritem in $error){
write-output "Error Data:" $erritem.TargetObject | out-file -Append "$finalstr2\GCI- ERRORS-backup.txt"
}
$error.Clear()
}
}
}