My aim is to compare two directories exactly - including the structure of the directories and sub-directories.
I need this, because I want to monitor if something in the folder E:\path2 was changed. For this case a copy of the full folder is in C:\path1. If someone changes something it has to be done in two directories.
It is important for us, because if something is changed in the directory (accidentally or not) it could break down other functions in our infrastructure.
This is the script I've already written:
# Compare files for "copy default folder"
# This Script compares the files and folders which are synced to every client.
# Source: https://mcpmag.com/articles/2016/04/14/contents-of-two-folders-with-powershell.aspx
# 1. Compare content and Name of every file recursively
$SourceDocsHash = Get-ChildItem -recurse –Path C:\path1 | foreach {Get-FileHash –Path $_.FullName}
$DestDocsHash = Get-ChildItem -recurse –Path E:\path2 | foreach {Get-FileHash –Path $_.FullName}
$ResultDocsHash = (Compare-Object -ReferenceObject $SourceDocsHash -DifferenceObject $DestDocsHash -Property hash -PassThru).Path
# 2. Compare name of every folder recursively
$SourceFolders = Get-ChildItem -recurse –Path C:\path1 #| where {!$_.PSIsContainer}
$DestFolders = Get-ChildItem -recurse –Path E:\path2 #| where {!$_.PSIsContainer}
$CompareFolders = Compare-Object -ReferenceObject $SourceFolders -DifferenceObject $DestFolders -PassThru -Property Name
$ResultFolders = $CompareFolders | Select-Object FullName
# 3. Check if UNC-Path is reachable
# Source: https://stackoverflow.com/questions/8095638/how-do-i-negate-a-condition-in-powershell
# Printout, if UNC-Path is not available.
if(-Not (Test-Path \\bb-srv-025.ftscu.be\DIP$\Settings\ftsCube\default-folder-on-client\00_ftsCube)){
$UNCpathReachable = "UNC-Path not reachable and maybe"
}
# 4. Count files for statistics
# Source: https://stackoverflow.com/questions/14714284/count-items-in-a-folder-with-powershell
$count = (Get-ChildItem -recurse –Path E:\path2 | Measure-Object ).Count;
# FINAL: Print out result for check_mk
if($ResultDocsHash -Or $ResultFolders -Or $UNCpathReachable){
echo "2 copy-default-folders-C-00_ftsCube files-and-folders-count=$count CRITIAL - $UNCpathReachable the following files or folders has been changed: $ResultDocs $ResultFolders (none if empty after ':')"
}
else{
echo "0 copy-default-folders-C-00_ftsCube files-and-folders-count=$count OK - no files has changed"
}
I know the output is not perfect formatted, but it's OK. :-)
This script spots the following changes successfully:
create new folder or new file
rename folder or file -> it is shown as error, but the output is empty. I can live with that. But maybe someone sees the reason. :-)
delete folder or file
change file content
This script does NOT spot the following changes:
move folder or file to other sub-folder. The script still says "everything OK"
I've been trying a lot of things, but could not solve this.
Does anyone can help me how the script can be extended to spot a moved folder or file?
I think your best bet is to use the .NET FileSystemWatcher class. It's not trivial to implement an advanced function that uses it, but I think it will simplify things for you.
I used the article Tracking Changes to a Folder Using PowerShell when I was learning this class. The author's code is below. I cleaned it up as little as I could stand. (That publishing platform's code formatting hurts my eyes.)
I think you want to run it like this.
New-FileSystemWatcher -Path E:\path2 -Recurse
I could be wrong.
Function New-FileSystemWatcher {
[cmdletbinding()]
Param (
[parameter()]
[string]$Path,
[parameter()]
[ValidateSet('Changed', 'Created', 'Deleted', 'Renamed')]
[string[]]$EventName,
[parameter()]
[string]$Filter,
[parameter()]
[System.IO.NotifyFilters]$NotifyFilter,
[parameter()]
[switch]$Recurse,
[parameter()]
[scriptblock]$Action
)
$FileSystemWatcher = New-Object System.IO.FileSystemWatcher
If (-NOT $PSBoundParameters.ContainsKey('Path')){
$Path = $PWD
}
$FileSystemWatcher.Path = $Path
If ($PSBoundParameters.ContainsKey('Filter')) {
$FileSystemWatcher.Filter = $Filter
}
If ($PSBoundParameters.ContainsKey('NotifyFilter')) {
$FileSystemWatcher.NotifyFilter = $NotifyFilter
}
If ($PSBoundParameters.ContainsKey('Recurse')) {
$FileSystemWatcher.IncludeSubdirectories = $True
}
If (-NOT $PSBoundParameters.ContainsKey('EventName')){
$EventName = 'Changed','Created','Deleted','Renamed'
}
If (-NOT $PSBoundParameters.ContainsKey('Action')){
$Action = {
Switch ($Event.SourceEventArgs.ChangeType) {
'Renamed' {
$Object = "{0} was {1} to {2} at {3}" -f $Event.SourceArgs[-1].OldFullPath,
$Event.SourceEventArgs.ChangeType,
$Event.SourceArgs[-1].FullPath,
$Event.TimeGenerated
}
Default {
$Object = "{0} was {1} at {2}" -f $Event.SourceEventArgs.FullPath,
$Event.SourceEventArgs.ChangeType,
$Event.TimeGenerated
}
}
$WriteHostParams = #{
ForegroundColor = 'Green'
BackgroundColor = 'Black'
Object = $Object
}
Write-Host #WriteHostParams
}
}
$ObjectEventParams = #{
InputObject = $FileSystemWatcher
Action = $Action
}
ForEach ($Item in $EventName) {
$ObjectEventParams.EventName = $Item
$ObjectEventParams.SourceIdentifier = "File.$($Item)"
Write-Verbose "Starting watcher for Event: $($Item)"
$Null = Register-ObjectEvent #ObjectEventParams
}
}
I don't think any example I've found online tells you how to stop watching the filesystem. The simplest way is to just close your PowerShell window. But I always seem to have 15 tabs open in each of five PowerShell windows, and closing one of them is a nuisance.
Instead, you can use Get-Job to get the Id of registered events. Then use Unregister-Event -SubscriptionId n to, well, unregister the event, where 'n' represents the number(s) you find in the Id property of Get-Job..
So basically you want to synchronize the two folders and note all the changes made on that:
I would suggest you to use
Sync-Folder Script
Or
FreeFile Sync.
Related
I am trying to create a reusable log function for a project I am working on in PowerShell. I am a novice with PowerShell so I am having problems figuring out why my function is not producing expected results. I am attempting to create a function that send NTFS/ACL details to a log file. This function will be incorporated in to a larger script that will change some NTFS/ACL/ACE folder rights. I have excluded some of the other code for simplification (changing rights).
Below is my current stripped down code. When it runs, it creates the log file but the file is empty. If I move the line of code that creates the log inside the log function, it creates the log file with data but it is not formatted correctly - it writes the heading (object attribute names) on one line, then the data, then then a new line with the heading, then the data. I want it to write the heading, then a line of data, line of data, .... Before I created a function for this, it worked as expected. I am a novice at PowerShell so I may not understand how to pass info in and out of the function. My Code:
#variables
$rootDir = "\\server1\share1"
$logDir = "c:\temp"
$date = (Get-Date -Format "MMddyyyy-HHmm")
$logData =#()
#My Logging Function
Function CreateLog {
#create and object to store attributes
$logObj = New-Object -TypeName psobject -Property #{
FolderPath = $Folder.Fullname
IdentityReference = $ACL.IdentityReference.ToString()
folder = $Folder[0]
FileSystemRights = $ACL.FileSystemRights.ToString()
InheritanceFlags = $ACL.InheritanceFlags.ToString()
}
$Folders=(Get-ChildItem -Path $rootDir -Directory -Recurse)
foreach ($Folder in $Folders){
$ACLs = get-acl $Folder.fullname | ForEach-Object {$_.Access}
Foreach ($ACL in $ACLs){
CreateLog
if($ACL.FileSystemRights -eq "FullControl"){
Write-Host "DO SOMETHING"
}
}
$logData | select folder,IdentityReference,FileSystemRights,InheritanceFlags,FolderPath | Format-Table -Wrap | Out-File $logDir\testlog-$date.log -Append
I assume you're looking for something like this:
#variables
$rootDir = "\\server1\share1"
$logDir = "c:\temp"
$date = Get-Date -Format "MMddyyyy-HHmm"
#My Logging Function
Function CreateLog {
param(
[Parameter(Mandatory)]
[System.Security.AccessControl.FileSystemAccessRule] $AccessRule,
[Parameter(Mandatory)]
[string] $Folder
)
[pscustomobject]#{
FolderPath = $Folder
IdentityReference = $AccessRule.IdentityReference
FileSystemRights = $AccessRule.FileSystemRights
InheritanceFlags = $AccessRule.InheritanceFlags
}
}
Get-ChildItem -Path $rootDir -Directory -Recurse | ForEach-Object {
foreach($rule in ($_ | Get-Acl).Access) {
CreateLog -AccessRule $rule -Folder $_.FullName
if($rule.FileSystemRights.HasFlag([Security.AccessControl.FileSystemRights]::FullControl)) {
# Do something here
}
}
} | Export-Csv $logDir\testlog-$date.log -NoTypeInformation
Your current code has a few syntax errors (missing }), your function seem to be assigning the objects to $logObj but then $logObj is never outputted from it, hence producing no output.
Your $logData array is defined but never used, leaving aside is not needed at all in this case. Your function should, ideally, take the arguments for the Access Rules and Path, see Functions with Parameters to learn more.
Format-Table as well as the other Format-* cmdlets should be primarily used for console display, should not be used for exporting data. In this case you should use Export-Csv, this way your data preserves its structure and can be imported back at ease, be filtered and be sorted.
It is a file organiser script I wrote for myself. For a specific purpose of mine. Whenever I try to run it It runs and closes off. But the move operation is not happening. The below comments may help you understand what the code is doing.Please help me on what am i doing wrong here. I am extremely new to Powershell Scripting.
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
# Global variable declarations
$global:pathsFromConfig = Get-Content -Path $PSScriptRoot"\MoverPaths.txt"
$global:categoriesFromConfig = Get-Content -Path $PSScriptRoot"\MoverCategories.txt"
$global:categryHash = #{}
# Method call to read configs, create dirs, & move files
readCreateAndMoveFiles
# Method definition
function readCreateAndMoveFiles{
# Reads categories config.txt and splits them line by line
# Adds each line as a key value pair to a hashtable
foreach($category in $categoriesFromConfig)
{
$temp = $category -split ":"
$categryHash.add($temp[0].trim().toString(),($temp[1]).trim().toString())
}
# For each category in the hash table, calls create directory method, and then moves the files based on current category
foreach($hashItem in $categryHash.GetEnumerator()){
# Creates a directory with the Hash Key
Foreach($pathToMonitor in $pathsFromConfig){
$categoryFullPath = $pathToMonitor+$hashItem.Name
createDirectory($categoryFullPath)
# Moves files into that directory
Set-Location -Path $pathToMonitor
$extentions = $hashItem.Value
Get-Item $extentions | Move-Item -Destination $categoryFullPath
$categoryFullPath = ""
}
}
}
# Method Definition
function createDirectory ($categoryName)
{
if(Test-Path -Path $categoryName)
{
# Directory already Exists!
}
else
{
# Creates Directory
md $categoryName
}
}
The config files are hereby:
MoverCategories.txt
Images:*.jpg,*.jpeg,*.png,*.tiff,*.raw,*.heic,*.gif,*.svg,*.eps,*.ico
Documents:*.txt,*.pdf,*.doc,*.docx,*.xls,*.xlsx,*.ppt,*.pptx,*.html,*.xls,*.csv,*.rtx
MoverPaths.txt
D:\Downloads\
Found a way to do this. Thanks for all of your input. Now the script moves files. Instead of sending all extentions in a single shot, i made it into an array and sent it one by one. Now it works fine. If you guys could help me reduce the time of execution that would be great.But the code works now I am happy.
foreach($hashItem in $categryHash.GetEnumerator()){
# Creates a directory with the Hash Key
Foreach($pathToMonitor in $pathsFromConfig){
$categoryFullPath = $pathToMonitor+$hashItem.Name
createDirectory($categoryFullPath)
# Moves files into that directory
[String[]]$extentions = #()
$extentions = $hashItem.Value -split ','
foreach($string in $extentions)
{
Get-Item $pathToMonitor\* -Include $string | Move-Item -Destination $categoryFullPath
}
}
}
Try this
#specify path(s)
$path = "$env:USERPROFILE\Downloads"
## this is make an array of the extensions in the foloder
$extensions = Get-ChildItem -Path $path | Select-Object -Unique -Property #{label = 'ext'
expression = { $_.Extension.substring(1) }
}
## this function will
function New-FoldersByName {
param
(
[Parameter(Mandatory = $true, ValueFromPipeline = $true, HelpMessage = 'Data to process')]
$InputObject
)
process {
Set-Location $path
if (!(Test-Path -PathType Container $InputObject )) {
New-Item -ItemType directory -Name $InputObject.ext -WhatIf
Write-Host -Message "A folder named $($InputObject.ext) does not exist. Creating..."
}
else {
Write-Host -Message "A folder named $($InputObject.ext) already exists. Skipping..."
}
}
}
##this is a reuseable function to moves items in a folder into a subfolder named after the files extension
## if extension is .exe the file with be moved to ./EXE/filename.exe
function Move-ItemsByName {
param
(
[Parameter(Mandatory = $true, ValueFromPipeline = $true, HelpMessage = 'Data to process')]
$InputObject
)
process {
Set-Location -Path $path
Move-Item -Path ('*.{0}' -f $InputObject.ext) -Destination ('{0}' -f $InputObject.ext) -WhatIf
}
}
$extensions | New-FoldersByName
$extensions | Move-ItemsByName
I have a script that copies the files from one server to another.
I have basically 3 different server locations, that I want to copy from , and create on another server, a folder for each of the location from the source and the contents inside it.
I made that, but I declared a variable for each source and each folder/destination.
I want to create one variable, and from that it should automatically get each location of the source, and copy everything in the correct location.
Would defining $server= "\path1 , \path2, \path3 " do it and it would go into a foreach loop? and go through each part of the path and copy and paste?
If so, how can I define the destination if I have 1 folder with 3 subfolders each corresponding to one source.
for example \path1 should alwasy put items in the path1destination , \path2 should always put items in the path2destination and so on. basically I want somehow to correlate that for each source path to have a specific path destination and everything should use as less variables as possible.
Anyone can provide and ideas on how to tackle this ? My code works but I had to define $path1 , $path2 , $path3 and so on, and then go for a loop on each, which is great but I need to make it clean and less lines of code .
$server1 = "C:\Users\nicolae.calimanu\Documents\B\"
$server2 = "C:\Users\nicolae.calimanu\Documents\A\" # UNC Path.
$datetime = Get-Date -Format "MMddyyyy-HHmmss"
$server3 = "C:\Users\nicolae.calimanu\Documents\C\" # UNC Path.
foreach ($server1 in gci $server1 -recurse)
{
Copy-Item -Path $server1.FullName -Destination $server2
}
ForEach ( $server2 in $server2 ) {
$curDateTime = Get-Date -Format yyyyMMdd-HHmmss
Get-ChildItem $server2 -Recurse |
Rename-Item -NewName {$_.Basename + '_' + $curDateTime + $_.Extension }
}
foreach ($server2 in gci $server2 -Recurse)
{
Move-Item -path $server2 -destination "C:\Users\nicolae.calimanu\Documents\C"
}
Use a hashtable to create a key-value store for each source and destination. Like so,
# Create entries for each source and destination
$ht = #{}
$o = new-object PSObject -property #{
from = "\\serverA\source"
to = "\\serverB\destination" }
$ht.Add($o.from, $o)
$o = new-object PSObject -property #{
from = "\\serverC\source"
to = "\\serverB\destination2" }
$ht.Add($o.from, $o)
$o = new-object PSObject -property #{
from = "\\servera\source2"
to = "\\serverC\destination" }
$ht.Add($o.from, $o)
# Iterate the collection. For demo, print the copy commands
foreach($server in $ht.keys) { $cmd = $("copy-item {0} {1}" -f $ht.Item($server).from, $ht.Item($server).to); $cmd }
# Sample output
copy-item \\serverA\source \\serverB\destination
copy-item \\servera\source2 \\serverC\destination
copy-item \\serverC\source \\serverB\destination2
I am trying to join multiple log files into a single archive file, then move that archive file to another location, in an effort to both clean up old log files, and save hard drive space. We have a bunch of tools that all log to the same root, with a per-tool folder for their logs. (E.g.,
C:\ServerLogs
C:\ServerLogs\App1
C:\ServerLogs\2ndApp
each of which will have log files inside, like
C:\ServerLogs\App1\June1.log
C:\ServerLogs\App1\June2.log
C:\ServerLogs\2ndApp\June1.log
C:\ServerLogs\2ndApp\June2.log
I want to go into each of these subfolders, archive up all the files older than 5 days, then move the archive to another (long-term storage) drive and delete the now-zipped files. The tools I'm using are PowerShell and 7zip. The below code is using test locations.
I have cobbled together two scripts from various sources online, over the course of two full shifts, but neither one works right. Here's the first:
# Alias for 7-zip
if (-not (test-path "$env:ProgramFiles\7-Zip\7z.exe")) {throw "$env:ProgramFiles\7-Zip\7z.exe needed"}
set-alias 7zip "$env:ProgramFiles\7-Zip\7z.exe"
$Days = 5 #minimum age of files to archive; in other words, newer than this many days ago are ignored
$SourcePath = C:\WorkingFolder\FolderSource\
$DestinationPath = C:\Temp\
$LogsToArchive = Get-ChildItem -Recurse -Path $SourcePath | Where-Object {$_.lastwritetime -le (get-date).addDays(-$Days)}
$archive = $DestinationPath + $now + ".7z"
#endregion
foreach ($log in $LogsToArchive) {
#define Args
$Args = a -mx9 $archive $log
$Command = 7zip
#write-verbose $command
#invoke the command
invoke-expression -command $Command $Args
The problem with this one is that I get errors trying to invoke the expression. I've tried restructuring it, but then I get errors because my $Args have an "a"
So I abandoned this method (despite it being my preferred), and tried this set.
#region Params
param(
[Parameter(Position=0, Mandatory=$true)]
[ValidateScript({Test-Path -Path $_ -PathType 'container'})]
[System.String]
$SourceDirectory,
[Parameter(Position=1, Mandatory=$true)]
[ValidateNotNullOrEmpty()]
[System.String]
$DestinationDirectory
)
#endregion
function Compress-File{
#region Params
param(
[Parameter(Position=0, Mandatory=$true)]
[ValidateScript({Test-Path -Path $_ -PathType 'leaf'})]
[System.String]
$InputFile,
[Parameter(Position=1, Mandatory=$true)]
[ValidateNotNullOrEmpty()]
[System.String]
$OutputFile
)
#endregion
try{
#Creating buffer with size 50MB
$bytesGZipFileBuffer = New-Object -TypeName byte[](52428800)
$streamGZipFileInput = New-Object -TypeName System.IO.FileStream($InputFile,[System.IO.FileMode]::Open,[System.IO.FileAccess]::Read)
$streamGZipFileOutput = New-Object -TypeName System.IO.FileStream($OutputFile,[System.IO.FileMode]::Create,[System.IO.FileAccess]::Write)
$streamGZipFileArchive = New-Object -TypeName System.IO.Compression.GZipStream($streamGZipFileOutput,[System.IO.Compression.CompressionMode]::Compress)
for($iBytes = $streamGZipFileInput.Read($bytesGZipFileBuffer, 0,$bytesGZipFileBuffer.Count);
$iBytes -gt 0;
$iBytes = $streamGZipFileInput.Read($bytesGZipFileBuffer, 0,$bytesGZipFileBuffer.Count)){
$streamGZipFileArchive.Write($bytesGZipFileBuffer,0,$iBytes)
}
$streamGZipFileArchive.Dispose()
$streamGZipFileInput.Close()
$streamGZipFileOutput.Close()
Get-Item $OutputFile
}
catch { throw $_ }
}
Get-ChildItem -Path $SourceDirectory -Recurse -Exclude "*.7z"|ForEach-Object{
if($($_.Attributes -band [System.IO.FileAttributes]::Directory) -ne [System.IO.FileAttributes]::Directory){
#Current file
$curFile = $_
#Check the file wasn't modified recently
if($curFile.LastWriteTime.Date -le (get-date).adddays(-5)){
$containedDir=$curFile.Directory.FullName.Replace($SourceDirectory,$DestinationDirectory)
#if target directory doesn't exist - create
if($(Test-Path -Path "$containedDir") -eq $false){
New-Item -Path "$containedDir" -ItemType directory
}
Write-Host $("Archiving " + $curFile.FullName)
Compress-File -InputFile $curFile.FullName -OutputFile $("$containedDir\" + $curFile.Name + ".7z")
Remove-Item -Path $curFile.FullName
}
}
}
This actually seems to work, insofar as it creates individual archives for each eligible log, but I need to "bundle" up the logs into one mega-archive, and I can't seem to figure out how to recurse (to get sub-level items) and do a foreach (to confirm age) without having that foreach produce individual archives.
I haven't even gotten into the Move and Delete phase, because I can't seem to get the archiving stage to work properly, but I certainly don't mind grinding away at that once this gets figured out (I've already spent two full days trying to figure this one!).
I greatly appreciate any and all suggestions! If I've not explained something, or been a bit unclear, please let me know!
EDIT1: Part of the requirement, which I completely forgot to mention, is that I need to keep the structure in the new location. So the new location will have
C:\ServerLogs --> C:\Archive\
C:\ServerLogs\App1 --> C:\Archive\App1
C:\ServerLogs\2ndApp --> C:\Archive\2ndApp
C:\Archive
C:\Archive\App1\archivedlogs.zip
C:\Archive\2ndApp\archivedlogs.zip
And I have absolutely no idea how to specify that the logs from App1 need to go to App1.
EDIT2: For this latter part, I used Robocopy - It maintains the folder structure, and if you feed it ".zip" as an argument, it'll only do the .zip files.
this line $Args = a -mx9 $archive $log likely needs to have the right side value wrapped in double quotes OR each non-variable wrapped in quotes with a comma between each so that you get an array of args.
another method would be to declare an array of args explicitly. something like this ...
$ArgList = #(
'a'
'-mx9'
$archive
$log
)
i also recommend you NOT use an automatic $Var name. take a look at Get-Help about_Automatic_Variables and you will see that $Args is one of those. you are strongly recommended NOT to use any of them for anything other than reading. writing to them is iffy. [grin]
I am still pretty new to scripting and "programming" at all. if you miss any information here let me know.
This is my working zip function:
$folder = "C:\zipthis\"
$destinationFilePath = "C:\_archive\zipped"
function create-7zip{
param([string] $folder,
[String] $destinationFilePath)
write-host $folder $destinationFilePath
[string]$pathToZipExe = "C:\Program Files (x86)\7-Zip\7zG.exe";
[Array]$arguments = "a", "-tzip", "$destinationFilePath", "$folder";
& $pathToZipExe $arguments;
}
Get-ChildItem $folder | ? { $_.PSIsContainer} | % {
write-host $_.BaseName $_.Name;
$dest= [System.String]::Concat($destPath,$_.Name,".zip");
(create-7zip $_.FullName $dest)
}
create-7zip $folder $destinationFilePath
now I want him to zip special folders which I already sorted out :
get-childitem "C:\zipme\" | where-Object {$_.name -eq "www" -or $_.name -eq "sql" -or $_.name -eq "services"}
This small function finds the 3 folders I need called www, sql and services. But I didn't manage to insert this into my zip function, so that exactly this folders are zipped and put into C:\_archive\zipped
Because a string is used instead of an array, he tried always to look for a folder called wwwsqlservice which is not there. I tried to put an array using #(www,sql,services) but i had no success, so whats the right way, if there is one?
It should compatible with powershell 2.0, no ps3.0 cmdlets or functions please.
thanks in advance!
Here's a really simple example of what you want to do, removed from the context of your function. It assumes that your destination folders already exist (You can just use Test-Path and New-Item to create them if they don't), and that you're using 7z.exe.
$directories = #("www","sql","services")
$archiveType = "-tzip"
foreach($dir in $directories)
{
# Use $dir to update the destination each loop to prevent overwrites!
$sourceFilePath = "mySourcePath\$dir"
$destinationFilePath = "myTargetPath\$dir"
cmd /c "$pathToZipExe a $archiveType $destinationFilePath $sourceFilePath"
}
Overall it looks like you got pretty close to a solution, with some minor changes needed to support the foreach loop. If you're confident that create-7zip works fine for a single folder, you can substitute that for the cmd /c line above. Here's a listing of some handy example usages for 7zip on the command line.