script file, wait function until end of generation files - powershell

I have a script file (batch file) which generate three files in a specific folder. Then i have a ps1 file which copy / move the generated files to another server / folders. Separately, everything is working properly
I'd like if it's possible to merge this, and have a wait function between the two scripts. In fact launching the copy / move ps1 function, only when the three files was correctly generated.

The following assumes:
that the files are created and written in full in a single operation.
that it is the appearance of a *.zip file that signals that all files of interest have been created (though they may still in the process of being written to), as you've indicated in a later comment.
$inFolder = '.' # set to the folder of interest.
$outFolder = './out' # ditto
Write-Verbose -vb 'Waiting for a *.zip file to appear...'
while (-not (Test-Path "$inFolder/*.zip")) { Start-Sleep 1 }
# Get a list of all files.
$files = Get-ChildItem -File $inFolder
Write-Verbose -vb 'Waiting for all files to be written completely...'
$files | ForEach-Object {
do {
# Infer from the ability to obtain an exclusive lock that the file has
# has been written in its entirety.
try { [IO.File]::Open($_.FullName, 'Open', 'Read', 'None').Dispose(); return }
catch { Start-Sleep 1 }
} while ($true)
}
# Move the files elsewhere
Write-Verbose -vb 'Moving...'
$files | Move-Item -Destination $outFolder -WhatIf
Note: The -WhatIf common parameter in the last command above previews the operation. Remove -WhatIf once you're sure the operation will do what you want.

param(
[String]$sourceDirectory = "c:\tmp\001",
[String]$destDirectory = "c:\tmp\001"
)
Get-ChildItem $sourceDirectory | ? {
#this step wait while locks free
[bool]$flag
while (!$flag) {
try {
$FileStream = [System.IO.File]::Open($_,'Open','Write')
$FileStream.Close()
$FileStream.Dispose()
$flag = $true
}
catch{
Start-ScheduledTask -s 1
$null
}
}
$true
} | Copy-Item -Destination $destDirectory

Related

Move multiple files from folder A to folder B

We need to move .csv files from folder where the are stored down to external server using powershell.
this is what i've tried so far but for some reason i only get message not copying and name of the files:
$DestinationFolder = "C:\d1\"
$SourceFolder = "C:\s1\"
If (-not (Test-Path $DestinationFolder) ) {
New-Item -ItemType Directory -Force -Path $DestinationFolder
}
$EarliestModifiedTime = (Get-Date).AddMinutes(200).Date # get's current time + adds 30 min
$LatestModifiedTime = (Get-Date).Date
echo($DestinationFolder); # will check time
Get-ChildItem "C:\s1\*.*" |
ForEach-Object {
if ( ($_.CreationTime -ge $EarliestModifiedTime) -and ($_.CreationTime -lt $LatestModifiedTime) ){ # i.e., all day yesterday
Copy-Item $_ -Destination $DestinationFolder -Force
Write-Host "Copied $_" }
else {
Write-Host "Not copying $_"
}
}
does it work if you simplify it and just try for a test (e.g. pick one file rather than trying to run for a delta of every 30 mins / last day)? Just thinking first you need to see if the problem is with accessing (or the formatting) of your source / destination directories or the delta logic itself. Perhaps some more error conditions would help....

How to pipe Rename-Item into Move-Item (powershell)

I'm in the process of writing up a PowerShell script that can take a bunch of .TIF images, rename them, and place them in a new folder structure depending on the original file name.
For example, a folder containing the file named:
ABC-ALL-210316-0001-3001-0001-1-CheckInvoice-Front.TIF
would be renamed to "00011CIF.TIF", and placed in the following folder:
\20220316\03163001\
I've been trying to put together a code to perform this task, and I got one to work where I had two different "ForEach" methods. One would do a bunch of file renaming to remove "-" and shorten "CheckInvoiceFront" to "CIF" and such. Then the second method would again pull all .TIF images, create substrings of the image names, and create folders from those substrings, and then move the image to the new folder, shortening the file name. Like I said, it worked... but I wanted to combine the ForEach methods into one process. However, each time I try to run it, it fails for various reasons... I've tried to change things around, but I just can't seem to get it to work.
Here's the current (non-working) code:
# Prompt user for directory to search through
$sorDirectory = Read-Host -Prompt 'Input source directory to search for images: '
$desDirectory = Read-Host -Prompt 'Input target directory to output folders: '
Set-Location $sorDirectory
# Check directory for TIF images, and close if none are found
Write-Host "Scanning "$sorDirectory" for images... "
$imageCheck = Get-ChildItem -File -Recurse -Path $sorDirectory -include '*.tif'
$imageCount = $imageCheck.count
if ($imageCount -gt 0) {
Write-Host "Total number of images found: $imageCount"
""
Read-Host -Prompt "Press ENTER to continue or CTRL+C to quit"
$count1=1;
# Rename all images, removing "ABCALL" from the start and inserting "20", and then shorten long filetype names, and move files to new folders with new names
Clear-Host
Write-Host "Reformatting images for processing..."
""
Get-ChildItem -File -Recurse -Path $sorDirectory -include '*.tif' |
ForEach-Object {
Write-Progress -Activity "Total Formatted Images: $count1/$imageCount" -Status "0--------10%--------20%--------30%--------40%--------50%--------60%--------70%--------80%--------90%-------100" -CurrentOperation $_ -PercentComplete (($count1 / $imageCount) * 100)
Rename-Item $_ -NewName $_.Name.Replace("-", "").Replace("ABCALL", "20").Replace("CheckInvoiceFront", "CIF").Replace("CheckInvoiceBack", "CIB").Replace("CheckFront", "CF").Replace("CheckBack", "CB") |Out-Null
$year = $_.Name.SubString(0, 4)
$monthday = $_.Name.Substring(4,4)
$batch = $_.Name.SubString(12, 4)
$fulldate = $year+$monthday
$datebatch = $monthday+$batch
$image = $_.Name.SubString(16)
$fullPath = "$desDirectory\$fulldate\$datebatch"
if (-not (Test-Path $fullPath)) { mkdir $fullPath |Out-Null }
Move-Item $_ -Destination "$fullPath\$image" |Out-Null
$count1++
}
# Finished
Clear-Host
Write-Host "Job complete!"
Timeout /T -1
}
# Closes if no images are found (likely bad path)
else {
Write-Host "There were no images in the selected folder. Now closing..."
Timeout /T 10
Exit
}
Usually this results in an error stating that it's can't find the path of the original file name, as if it's still looking for the original non-renamed image. I tried adding some other things, but then it said I was passing null values. I'm just not sure what I'm doing wrong.
Note that if I take the everything after the "Rename-Item" (starting with "$year =") and have that in a different ForEach method, it works. I guess I just don't know how to make the Rename-Item return its results back to "$_" before everything else tries working on it. I tried messing around with "-PassThru" but I don't think I was doing it right.
Any suggestions?
As Olaf points out, situationally you may not need both a Rename-Item and a Move-Item call, because Move-Item can rename and move in single operation.
That said, Move-Item does not support implicit creation of the target directory to move a file to, so in your case you do need separate calls.
You can use Rename-Item's -PassThru switch to make it output a System.IO.FileInfo instance (or, if a directory is being renamed, a System.IO.DirectoryInfo instance) representing the already renamed file; you can directly pass such an instance to Move-Item via the pipeline:
Get-ChildItem -File -Recurse -Path $sorDirectory -include '*.tif' |
ForEach-Object {
# ...
# Use -PassThru with Rename-Item to output a file-info object describing
# the already renamed file.
$renamedFile = $_ | Rename-Item -PassThru -NewName $_.Name.Replace("-", "").Replace("ABCALL", "20").Replace("CheckInvoiceFront", "CIF").Replace("CheckInvoiceBack", "CIB").Replace("CheckFront", "CF").Replace("CheckBack", "CB")
# ...
# Pass $renamedFile to Move-Item via the pipeline.
$renamedFile | Move-Item -Destination "$fullPath\$image"
# ...
}
As for your desire to:
make the Rename-Item return its results back to "$_"
While PowerShell doesn't prevent you from modifying the automatic $_ variable, it is better to treat automatic variables as read-only.
Therefore, a custom variable is used above to store the output from Rename-Item -PassThru
You need -passthru and -destination:
rename-item file1 file2 -PassThru | move-item -Destination dir1

Perform file verification check with Copy-Item

The following script creates a folder named the specified date on the servers in servers.txt, then copies the folder from which the script is run to that folder.
For example, it creates the folder "3-22-15 (night)" on SERVER1, SERVER2, etc., then copies "Directory Containing the Script" to "3-22-15 (night)".
$CurrentLocation = get-location
$DeploymentDate = "3-22-15 (night)"
Foreach($server in get-content "servers.txt"){
New-Item -ErrorAction SilentlyContinue -ItemType directory -Path \\$server\C$\Deployments\$DeploymentDate
copy-item -Path $CurrentLocation -Destination \\$server\C$\Deployments\$DeploymentDate\ -ErrorAction SilentlyContinue -recurse
}
How do I modify the script to include file verification for each file that is copied to \\$server\C$\Deployments\$DeploymentDate\?
I would like it to output an error message with the file that does not pass the verification check, but to continue copying.
I was going to try something like this:
function SafeCopy ($SourcePath,$DestinationPath,$SourceFileName) {
#MD5 Check Function
function Check-MD5 ($FilePath) {
$md5=New-Object -TypeName System.Security.Cryptography.MD5CryptoServiceProvider
$hash=[System.BitConverter]::ToString($md5.ComputeHash([System.IO.File]::ReadAllBytes($FilePath)))
Return $hash
} # EndOf function Check-MD5
$MD5Source=Check-MD5 $SourcePath
Copy-Item $SourcePath\$SourceFileName $DestinationPath
$MD5Destination=Check-MD5 $DestinationPath
if (Test-Path $DestinationPath) {
if ($MD5Destination -match $MD5Source) {
Write-Host "`nThe file `"$SourcePath`" has been copied in `"$DestinationPath`" successfully.`n" -ForegroundColor DarkGreen
} else {
Write-Host "`nThe file `"$SourcePath`" has been copied in `"$DestinationPath`" but the CRC check failed!`n" -ForegroundColor DarkRed
}
} else {
Write-Host "`nThe file `"$SourcePath`" has not been copied in `"$DestinationPath`"!`n" -ForegroundColor DarkRed
}
} # EndOf function SafeCopy
But I'm not sure how to implement it.
I am not a talented scripter nor do I play one on the radio but I've put this together and it seems to work. As mentioned, the larger the filesize being compared, the slower it goes.
Param
(
[parameter(Mandatory=$true,Position=1,ValueFromPipeLine=$true)][string]$source,
[parameter(Mandatory=$true,Position=2,ValueFromPipeLine=$true)][string]$dest
)
$countsource = #(Get-ChildItem -Path $source -Directory)
$countdest = #(Get-ChildItem -Path $dest -Directory)
IF($countsource = $countdest){
"$source equals $dest"
}
Else{
"$source does not match $dest"
}
Sample usage and output. TODO: Add folder item count compare, warn or break if the folders do not have the same number of items (subfolders and files).
So copy the folder then run Folder-Compare as a function once the copy is finished.
PS C:\Scripts> .\Folder-Compare.ps1 -source C:\Scripts\temp -dest C:\aaa
C:\Scripts\temp does not match C:\aaa

How many ways to check if a script was "successful" by using Powershell?

I am still very new and I have for example one script to backup some folders by zipping and copying them to a newly created folder.
Now I want to know if the zip and copy process was successful, by successful i mean if my computer zipped and copied it. I don't want to check the content, so I assume that my script took the right folders and zipped them.
Here is my script :
$backupversion = "1.65"
# declare variables for zip
$folder = "C:\com\services" , "C:\com\www"
$destPath = "C:\com\backup\$backupversion\"
# Create Folder for the zipped services
New-Item -ItemType directory -Path "$destPath"
#Define zip function
function create-7zip{
param([String] $folder,
[String] $destinationFilePath)
write-host $folder $destinationFilePath
[string]$pathToZipExe = "C:\Program Files (x86)\7-Zip\7zG.exe";
[Array]$arguments = "a", "-tzip", "$destinationFilePath", "$folder";
& $pathToZipExe $arguments;
}
Get-ChildItem $folder | ? { $_.PSIsContainer} | % {
write-host $_.BaseName $_.Name;
$dest= [System.String]::Concat($destPath,$_.Name,".zip");
(create-7zip $_.FullName $dest)
}
Now I can either check if in the parentfolder is a newly created folder by time.
Or i can check if there are zip folders in my subfolders I created.
What way would you suggest? I probably just know this ways, but there are a million way to do this.
Whats your idea? The only rule is , that powershell should be used.
thanks in advance
You could try using the Try and Catch method by wrapping the (create-7zip $_.FullName $dest) with a try and then catch any errors:
Try{ (create-7zip $_.FullName $dest) }
Catch{ Write-Host $error[0] }
This will Try the function create-7zip and write any the errors that many accrue to the shell.
One thing that can be tried is checking the $? variable for the status of the command.
$? stores the status of the last command run,
So for
create-7zip $_.FullName $dest
If you then echo out $? you will see either true or false.
Another option is the $error variable
You can also combine these in all sorts of ways (Or with the exception handling).
For example, run your command
foreach-object {
create-7zip $_.FullName $dest
if (!$?) {"$_.FullName $ErrorVariable" | out-file Errors.txt}
}
That script is more pseudocode for ideas than working code, but it should at least get you close to using it!

Create directory if it does not exist

I am writing a PowerShell script to create several directories if they do not exist.
The filesystem looks similar to this
D:\
D:\TopDirec\SubDirec\Project1\Revision1\Reports\
D:\TopDirec\SubDirec\Project2\Revision1\
D:\TopDirec\SubDirec\Project3\Revision1\
Each project folder has multiple revisions.
Each revision folder needs a Reports folder.
Some of the "revisions" folders already contain a Reports folder; however, most do not.
I need to write a script that runs daily to create these folders for each directory.
I am able to write the script to create a folder, but creating several folders is problematic.
Try the -Force parameter:
New-Item -ItemType Directory -Force -Path C:\Path\That\May\Or\May\Not\Exist
You can use Test-Path -PathType Container to check first.
See the New-Item MSDN help article for more details.
$path = "C:\temp\NewFolder"
If(!(test-path -PathType container $path))
{
New-Item -ItemType Directory -Path $path
}
Test-Path -PathType container checks to see if the path exists and is a directory. When it does not, it will create a new directory. If the path exists but is a file, New-Item will raise an error (you can overwrite the file by using the -force argument if you are risky).
[System.IO.Directory]::CreateDirectory('full path to directory')
This internally checks for directory existence, and creates one, if there is no directory. Just one line and native .NET method working perfectly.
Use:
$path = "C:\temp\"
If (!(test-path $path))
{
md $path
}
The first line creates a variable named $path and assigns it the string value of "C:\temp"
The second line is an If statement which relies on the Test-Path cmdlet to check if the variable $path does not exist. The not exists is qualified using the ! symbol.
Third line: If the path stored in the string above is not found, the code between the curly brackets will be run.
md is the short version of typing out: New-Item -ItemType Directory -Path $path
Note: I have not tested using the -Force parameter with the below to see if there is undesirable behavior if the path already exists.
New-Item -ItemType Directory -Path $path
The following code snippet helps you to create a complete path.
Function GenerateFolder($path) {
$global:foldPath = $null
foreach($foldername in $path.split("\")) {
$global:foldPath += ($foldername+"\")
if (!(Test-Path $global:foldPath)){
New-Item -ItemType Directory -Path $global:foldPath
# Write-Host "$global:foldPath Folder Created Successfully"
}
}
}
The above function split the path you passed to the function and will check each folder whether it exists or not. If it does not exist it will create the respective folder until the target/final folder created.
To call the function, use below statement:
GenerateFolder "H:\Desktop\Nithesh\SrcFolder"
I had the exact same problem. You can use something like this:
$local = Get-Location;
$final_local = "C:\Processing";
if(!$local.Equals("C:\"))
{
cd "C:\";
if((Test-Path $final_local) -eq 0)
{
mkdir $final_local;
cd $final_local;
liga;
}
## If path already exists
## DB Connect
elseif ((Test-Path $final_local) -eq 1)
{
cd $final_local;
echo $final_local;
liga; (function created by you TODO something)
}
}
When you specify the -Force flag, PowerShell will not complain if the folder already exists.
One-liner:
Get-ChildItem D:\TopDirec\SubDirec\Project* | `
%{ Get-ChildItem $_.FullName -Filter Revision* } | `
%{ New-Item -ItemType Directory -Force -Path (Join-Path $_.FullName "Reports") }
BTW, for scheduling the task please check out this link: Scheduling Background Jobs.
There are three ways I know to create a directory using PowerShell:
Method 1: PS C:\> New-Item -ItemType Directory -path "C:\livingston"
Method 2: PS C:\> [system.io.directory]::CreateDirectory("C:\livingston")
Method 3: PS C:\> md "C:\livingston"
From your situation it sounds like you need to create a "Revision#" folder once a day with a "Reports" folder in there. If that's the case, you just need to know what the next revision number is. Write a function that gets the next revision number, Get-NextRevisionNumber. Or you could do something like this:
foreach($Project in (Get-ChildItem "D:\TopDirec" -Directory)){
# Select all the Revision folders from the project folder.
$Revisions = Get-ChildItem "$($Project.Fullname)\Revision*" -Directory
# The next revision number is just going to be one more than the highest number.
# You need to cast the string in the first pipeline to an int so Sort-Object works.
# If you sort it descending the first number will be the biggest so you select that one.
# Once you have the highest revision number you just add one to it.
$NextRevision = ($Revisions.Name | Foreach-Object {[int]$_.Replace('Revision','')} | Sort-Object -Descending | Select-Object -First 1)+1
# Now in this we kill two birds with one stone.
# It will create the "Reports" folder but it also creates "Revision#" folder too.
New-Item -Path "$($Project.Fullname)\Revision$NextRevision\Reports" -Type Directory
# Move on to the next project folder.
# This untested example loop requires PowerShell version 3.0.
}
PowerShell 3.0 installation.
Here's a simple one that worked for me. It checks whether the path exists, and if it doesn't, it will create not only the root path, but all sub-directories also:
$rptpath = "C:\temp\reports\exchange"
if (!(test-path -path $rptpath)) {new-item -path $rptpath -itemtype directory}
I wanted to be able to easily let users create a default profile for PowerShell to override some settings, and ended up with the following one-liner (multiple statements yes, but can be pasted into PowerShell and executed at once, which was the main goal):
cls; [string]$filePath = $profile; [string]$fileContents = '<our standard settings>'; if(!(Test-Path $filePath)){md -Force ([System.IO.Path]::GetDirectoryName($filePath)) | Out-Null; $fileContents | sc $filePath; Write-Host 'File created!'; } else { Write-Warning 'File already exists!' };
For readability, here's how I would do it in a .ps1 file instead:
cls; # Clear console to better notice the results
[string]$filePath = $profile; # Declared as string, to allow the use of texts without plings and still not fail.
[string]$fileContents = '<our standard settings>'; # Statements can now be written on individual lines, instead of semicolon separated.
if(!(Test-Path $filePath)) {
New-Item -Force ([System.IO.Path]::GetDirectoryName($filePath)) | Out-Null; # Ignore output of creating directory
$fileContents | Set-Content $filePath; # Creates a new file with the input
Write-Host 'File created!';
}
else {
Write-Warning "File already exists! To remove the file, run the command: Remove-Item $filePath";
};
$mWarningColor = 'Red'
<#
.SYNOPSIS
Creates a new directory.
.DESCRIPTION
Creates a new directory. If the directory already exists, the directory will
not be overwritten. Instead a warning message that the directory already
exists will be output.
.OUTPUT
If the directory already exists, the directory will not be overwritten.
Instead a warning message that the directory already exists will be output.
.EXAMPLE
Sal-New-Directory -DirectoryPath '.\output'
#>
function Sal-New-Directory {
param(
[parameter(mandatory=$true)]
[String]
$DirectoryPath
)
$ErrorActionPreference = "Stop"
try {
if (!(Test-Path -Path $DirectoryPath -PathType Container)) {
# Sal-New-Directory is not designed to take multiple
# directories. However, we use foreach to supress the native output
# and substitute with a custom message.
New-Item -Path $DirectoryPath -ItemType Container | `
foreach {'Created ' + $_.FullName}
} else {
Write-Host "$DirectoryPath already exists and" `
"so will not be (re)created." `
-ForegroundColor $mWarningColor
}
} finally {
$ErrorActionPreference = "Continue"
}
}
"Sal" is just an arbitrary prefix for my own library. You could remove it or replace it with your own.
Another example (place here because it otherwise ruins stackoverflow syntax highlighting):
Sal-New-Directory -DirectoryPath ($mCARootDir + "private\")
Example, create a 'Reports' folder inside of the script's folder.
$ReportsDir = $PSScriptRoot + '\Reports'
$CreateReportsDir = [System.IO.Directory]::CreateDirectory($ReportsDir)